We've always travelled with our children but now they are teenagers (14 and 16) we give them more say so this summer I has a business trip to Sri Lanka which we could have built our summer holiday around but when we discussed it they said if we were having a big trip (we do a transcontinental trip for at least 4 weeks about every three years and do about 3 weeks in Europe in between) then they would rather go to New York and do a trip from there.
I echo what has been said about creating memories. I am acutely aware that in a few years they won't want to come with us so am enjoying while we can (although I still holidayed with my parents as an extra holiday when I was 21 if they went somewhere good).