Best guess is this is happening at busier times of day. For instance, evening in a home/apartment environment, or business hours in an office environment. Your voice packets are competing with millions of other peoples', and receive no priority. And, they really need that priority, being much more time sensitive.

As an IT Director, I wouldn't recommend depending upon voice over a public Internet at any price. I realize the above isn't very helpful, so let me throw in a little test to rule out the parts nearest to you.

First, find a path. Do this by opening a command prompt (in Windows 2000 or XP, Start, Run, type "cmd", and press Enter). Type:
tracert www.flyertalk.com
and press Enter. That'll show you the various routers (Internet traffic cops) your data passes through on the way to FlyerTalk. Pick one somewhere in the middle, and note the number. Let's say 206.71.65.42. You can then type:
ping 206.71.65.42
and press Enter. Note the time in milliseconds (ms). that's how long it takes for your data to reach that point. As a rule of thumb, anything over 90ms will start to become shaky for voice. However, these may go up and down. To run the test continuously, type:
ping -t 206.71.65.42
and press Enter. That'll keep going until you stop it. See if the times go up and down, or you start getting the dreaded asterisks (which mean nothing is getting there). If you do hit a problem area, try to isolate it. Do the same test with a closer (nearer to the beginning of the tracert test). Eventually, you'll be back to your cable modem (the first number in the tracert test)

Oh, and be considerate, after you've watched for a while, press <Ctrl>+<C> to stop the test. You don't want to continuously bombard some middleman with you data.

All that is hardly the "best" way of isolating the problem, but it's a relatively simple and free way to start the solution. Hope it helps a bit!