There is public bus service to/from the airport as follows:

Bus 204 east (est) bound, from the arrivals area on street level. Disembark at Dorval Station (Gare Dorval) and connect to Bus 211 east (est) bound. Depending upon the time of day, you may have to go through an underground viaduct to reach the 211, but on Sunday this won't be necessary. The 211 will take you to the Lionel Groulx Metro station. From there you can connect to anywhere downtown, but Atwater is closest and there are a number of dining and deli options in the adjoining Plaza Alexis Nihon, Forum and on St. Catherine Street. Cash fare is $2.50 and exact fare is required.

The trip downtown will take about an hour, depending on bus connections and a less frequent Sunday service, so I very much doubt you'll have enough time given your 3 hour layover. 18:30 on a Sunday is definitely going to limit your choices, as the best options at Marché Atwater will already have closed for the night. You could stay in the Dorval area and grab a bite nearby. There is a Maxi supermarket (open until 21:00) and a few dining options across from Dorval Station (Gare Dorval) that you can reach via the 204 bus.

Montreal Bus & Metro Info