Tokyo is not a city of world-class tourist attractions. The Rainbow bridge doesn't match up to the Golden Gate bridge nor is Sky Tree memorable like the Eiffel Tower. For me, Tokyo has always been a city of ever so many unexpected small delights, and these are best seen by walking around, often with no particular destination in mind, just as you are planning to do. So I think you've got absolutely the right idea in mind. I say this as someone who has been to Tokyo 50+ times.

You might, however, consider buying one of the small number of guides that give walking tours. Just be fully prepared to divert from the plan if you see something that interests you. If you get lost, don't worry. Sooner or later you will hit a rail or subway station. If not, most bus routes eventually pass one. And at no time, day or night, will you ever be in any physical danger; in that regard, Tokyo is a very safe city.

And incidentally, contrary to the advice in the post above, I see no need to take taxis anywhere unless you are totally lost and don't see a bus stop or are arriving/departing with baggage.