That's what we did for Japan Do this year. It was over Easter last year, picked the Easter Weekend again this year (obviously the actual dates differed). Some who attended last year can't make it this year, but them's the breaks and everyone understands.

If you want the 'do to be an annual thing, picking a date and sticking to it is the best way forward. Some might make it, some might look forward to the second, third or fourth iteration.

Eg. SIN Do is always the 2nd/3rd weekend of January, OzFest is always the memorial day weekend, same for the London Hyatt Do, and a couple of other regular ones. Japan Do, in its first two iterations would have been over the Easter Weekend

What might also help is if you keep the event sufficiently spaced from other major Do's so people have time to plan forward. And justify the additional flight and hotel costs to their spouse/partner/bank account.

Just my 2c as a 'do organizer.