Normandy is a big region, stretching from the outer reaches of Paris's western commuter belt to a rural landscape synonymous with dairy farming and cider production.
There are Intercity trains (i.e. non-TGV) which leave from two of the Paris stations: Saint-Lazare for Northern Normandy (Giverny, Rouen, Deauville, Trouville, Le Havre, Bayeux, Caen and the WWII sites) and the slightly obscure corner of Montparnasse known as Vaugirard, where trains head for Granville, on the west coast of the Manche peninsula, via the rural heartland of the region.
This should give you an idea of what is accessible by rail (although bear in mind that some branch lines may have a very limited service):
https://www.ter.sncf.com/normandie/g...-ter-normandie
This gives you an idea of the tourist sites accessible by rail (Google Translate can help...):
https://www.ter.sncf.com/normandie/l...s-touristiques
The same site will give you an idea of what tickets are available - whether point-to-point, or more flexible.