As a native Icelander I really wouldn't recommend going far out of your way just to visit Reykjavík. Sure, check out the city for a day if you are passing through but it really doesn't offer much of interest in my view. Iceland is about exploring the natural sights.