The hotel should have a water kettle in the room. If they don't just ask. They're used very, very, very frequently here.
To be honest, a stroller in shopping, mall areas etc. will be quite annoying. Seoul has tons of people and you're going to high traffic areas. You'll need a lot of patience to navigate a stroller. Also, for subways, you'll have to (sometimes) go out of your way for the elevator and end up at different exits from where you need to be. Having a stroller on the subway is no problem, though, as long as it's not crowded.
Taxis are very cheap here. And much more convenient for you with a baby. When you get a T-Money card, just load it up with a lot of cash. Maybe 50. Then you can pay for taxis with the t-money card as you take them. And continue to use T-Money on the subway.
For you, I'd just stick to Lotte World. You can take Subway Line 2 right there. I make this commute every day actually - just 35-40 minutes to Jamsil Station from Euljrio 1-Ga.
Well, with rice meals, you're in luck. Koreans eat rice 35 times a day. The best dish for you is ddeop bap. It's various meat sauteed with spicy sauce and pork, chicken etc. Also try Omu Rice, which is like rice inside a big omelet with donkatsu sauce on it. There are also a ton of soups in Korea. Their daily food is definitely not sweet. Think sesame oil and red pepper tastes.
You need to find a "kimbap shop." In Myeongdong, there are several. The hotel can direct you. They'll have pictures and an English menu. And they are the same price as normal kimbap shops throughout Seoul. This is where you can get the widest variety of Korean food. 40 dishes or so.