Just tried traveling with my EOS M (one week, multiple stops in Asia, mostly visiting friends.)
Not a winner; too slow and clumsy to replace a DSLR, too big to be pocketable or as inconspicuous/go-everywhere as my S95.
At least until I get a phone with decent optics, I don't see traveling without the S95 again.
If the M2 is enough better, I could definitely see bringing this again on trips where I'd otherwise bring my DSLR, and even without that there are some where right now I only bring my S95 and I'd bring the current M as a second camera.