Definitely skip taxi, that would make things too complicated and cost you some of the already little time you have in Paris. Otherwise I don't see a problem with your plan, I do side trip to Paris during long transfers quite often, it's totally doable.