In general, yes. The airport in Reykjavik (Keflavik) is compact and most of Icelandair's flights come in from the USA in the early morning, head out to Europe in the morning, are back in the early afternoon and then to the USA for arrivals in the evening.
You'll find shopping and cafés available, smoking rooms, bars and restroom facilities. None of those are large or comfortable, but most of the transfer times are under an hour and a half, so there's no point in getting comfortable because you'll be on your flight shortly.
Before the expansion, there was only room for 6 aircraft. Now there's room for maybe 15 aircraft if they squeeze in nicely.
6 years flying Icelandair and luckily never missed a connection personally. It does happen, generally due to delays from departure airports for coming into KEF, which will *sometimes* mean a delay leaving KEF for outbounds.
Travel well.