I think you may be a bit optimistic with your plans to be away for 3 weeks in 3 locations, without knowing how your baby is going to be. After 1 week you might wish you were about to go home. OTOH it may be ok if you are not doing anything except relaxing all day.

4-5 months can be a good time to travel because they aren't eating yet, however they may also have trouble with sleeping around this time. Maybe it's just us, but we wouldn't be leaving our baby in the room and going to the beach unless the room literally has direct access to the beach.