Personally, I find Narita city pretty boring. But, the OP probably has closer to 6-7 hours after you factor in immigration lines, check-in buffer, etc. Subtract another 2.5 hrs R/T for the NEX and we're into the 3-4 hr timeframe.

If it's your 1st time in TYO, I wouldn't do it. If you've been there many times and know exactly where to go, I'd go for it.