I've done this on layovers a lot shorter than that and think the experience is worth it. When Al Safwa rules were stricter I used to use it as a way to avoid having to visit Al Mourjan at all when flying on routes without F - the pool is typically wonderfully quiet, gym equipment is decent, there will be no wait for a shower and the temperature controls in the showers actually work, too! I've only ever seen one other person using the pool whilst I'm there, most of the time I'd have the entire pool to myself. Slightly surreal experience going for a swim whilst air side.