There is also a "Noctilien" night bus (bus N120), that goes through gares du Nord et de l'Est, Chatelet, Gares de Lyon et Austerlitz, Place d'Italie then to Orly-Sud on its way to Corbeil Essonnes. There is a departure at Chatelet around 4.30am (about 1/4 hr earlier at Gare du Nord/Est, and about 10 mins later at Gare de Lyon/Austerlitz), that arrives at Orly around 5.15am. There should be more info on it on the ratp.fr site.

For the evening flight, depending on where you are located in central Paris, it can be quicker to get on the metro to Villejuif Louis-Aragon (end of line 7) and then take the jetbus to Orly. The jetbus normally takes about 15 mins (I have never used it at rush hours, though). This would not work for the morning departure, however, as the first bus leaves Villejuif at 6.15am (and arrives at Orly at about 6.30am, which would be too late for a 7.10am departure).

Incidentally, these options are typically cheaper than the RER/Orlyval or Orlybus options.