OSL is a surprisingly confusing place. I would never risk a back to back. I had 3 hrs from NL and it was fine (even with some stupidity on my part) but I managed to wind up airside without boarding passes, was simply allowed to go into a service area and made my way back landslide without ever seeing passport control. Could have been arrested. Should have been really.
Anyway, the lounge is nice and AFAIK the other Norwegian airports don't have one. So a more relaxed connection would not be too bad. Norwegian are the handling agents for BA at OSL so maybe you can even check bags through? I don't know. HBO for me.