I'm not sure if it's considered off the beaten path anymore, but the Distillery District fits the bill for lovely/charming. It's a beautiful old area with nice galleries, good dining, cafes etc. There are no major chains, so everything there is definitely unique. It can be accessed via the 504 King streetcar.

http://www.thedistillerydistrict.com/frameset.html