I would highly recommend the World War 2 museum. That will take up a whole day. It's in the Warehouse District. There is also the Art Museum in City Park, Ogden Museum of Southern Art, Southern Food and Beverage Museum, Audubon Aquarium and Insectarium.
If you are not into museums, definitely head to Magazine St. and just start walking. Lots of amazing restaurants, shops, etc. to check out. Magazine St. extends from the Central Business District all the way to Audubon Park. Other cool streets to check out are Oak St. (take the St. Charles Ave. Streetcar) and St. Claude Ave. (take the new Rampart St. Streetcar to the end of Elysian Fields and start walking. You can also check out the very cool Crescent Park on the riverfront in the Bywater neighborhood.
If you are into biking take the new Lafitte Greenway from the French Quarter up to Mid-City and City Park. Mid-City has a great commercial district adjacent to the Greenway.
If you are into nature, either rent a car or set up a trip with a tour group to do a swamp tour. You could also head up River Road and visit some of the historic plantations if you are into history. I would highly recommend Whitney Plantation. Chalmette Battlefield is also interesting (site of the Battle of New Orleans).