Arundel as already noted, Chichester, Lewes, Chatham (for the Chatham Historic Dockyard), Canterbury - note for Canterbury, the fastest trains from London are now from St Pancras station (next to King's Cross), but it's still perfectly do-able to get there on a slower train from Victoria (it's also a bit cheaper). Eastbourne and Tunbridge Wells are newer towns (by British standards) but of interest.

If willing to change at Clapham Junction, or otherwise start from Waterloo station, then perhaps Salisbury, Romsey, Brockenhurst, Lymington, Portsmouth (for Portsmouth Historic Dockyard).

There's a lot out there depending upon what you're after.

You could take a look at the Days Out Guide website produced by the train companies, which might provide some ideas - it also shows various offers available to those who hold train tickets. Perhaps take a look at the interactive map at the bottom of this page, focussing on the counties of Hampshire, West and East Sussex, Kent and Surrey.

Both the National Trust and English Heritage look after many places of historic interest - the National Trust also owns and preserves places of natural beauty - their websites include details of all of these.

"The Beautiful South" / Visit South East England website might also provide inspiration.