Absolutely worth it, no question at all.
Transport makes getting to/from the airport pretty easy, so while it's not terribly close, it's not all that bad either.
Normally I tell people travelling from the US to stay awake on the plane and then crash at the hotel, but this is a MR - so there's no real point to that. Just suck it up and you'll be fine.
I say this because of the mention that you've always been fascinated by Tokyo/Japan - go for it, you won't be disappointed.
Personally I'd recommend Shinjuku over Shinagawa for someone with limited time, but near Tokyo station above Shinjuku.