There have been a number of threads here on the general topic of what to do with a layover of X hours at LHR. (Never been to LRH, but I suspect it's as lot quicker to get out of and back into, but not nearly as much to do in the area once you're out.) The consensus seems to be that 5 hours is about the minimum time for which it's worth going into London or somewhere else. I've done it with 4, and had plenty of time for lunch at a pub, but that was back when we didn't have to arrive as early for a flight as we should today.
I'd hold off on that decision until you see how long it takes you to get landside (should be quick, you should be eligible for FasTrack) and how energetic you both feel on the day (impossible for me to predict, perhaps also for you). It also depends on whether your luggage is checked through to Oslo. (It should be if both flights are on one ticket, might be even if they're not.) Other factors are whether you got your LHR-OSL boarding passes at EWR and how much time the two of you have already spent in London. If your flight is on time, you get into the UK fairly quickly, you're not too tired, you don't have to deal with luggage, you already have your boarding passes for the next flight, and London or one of the other places mentioned above is of interest, it's eminently do-able. If not, any lounge beats sitting at the gate, or go shopping and see what they're sampling at the World of Whisky shop in your terminal. I had a very pleasant chance to compare 12-, 15- and 21-year old Balvenie (if memory serves) once.