I've always loved the Musee National du Moyen Age on place Paul Painlevé
in the 5th (Cluny-Sorbonne metro stop is closest) It's in a 15th-century building, with a wonderful courtyard and architecture - and the bonus of Roman baths underneath. It's a great place to spend a morning or afternoon without the overwhelming-ness of the Louvre. Should be closed on Tuesdays, like the Louvre and other national museums. (City museums, like the Orsay, are closed on Mondays.) If you're planning on seeing more than 2 or 3 museums I'd get the "carte musee" museum pass - very useful for things like getting into the Louvre via the Passage Richelieu side entrance - no lines.

Best place for breakfast on a Sunday - Maison Korcasz in the Marais - the Jewish quarter bustles while the rest of the city is either asleep or in church.

I've always stayed in the 7th - nice and quiet - but might branch out this May as my significant other is a little pickier about hotels than I was 10 years ago.