I've only been to a couple, but really enjoyed Koishikawa Korakuen. Walked over from the Tokyo Dome where my wife was going to a quilt show and felt like an oasis from the cityscape. Very peaceful with a number of people eating their bag lunches scattered throughout.

Shinjuku Gyoen is quite nice for the different garden types incorporated. People watching here is fun, from the sketchers in the mother and child forest to the amateur photographers with telescope sized lens. Don't miss the attack carp! Even if the tea house isn't open, lots of vending machines around to pick up a hot milk tea and corn soup on a cold winter day.