The problem with having such a narrow vision of what is "worthwhile" is you may very well miss out on phenomenal deals. Capital One, for example, earlier this year had what I am convinced was the best deal ever for someone with your profile. They "matched" up to 100,000 of one's airline miles in a single program. Those capital one miles while not miles in the truest sense could be used for $1000 worth of paid flights/mileage running/hotels/rental cards/train tickets/taxis/anything that could arguably be travel related. Hardly chump change. Additionally you'll see other posters switching less needed airline miles to hotel miles or other items of value. So yes, you may be shortchanging yourself. It doesn't take *that* much work to keep track of your points, but to each his own.
That said, I'm not sure it is worth stockpiling for retirement or some such. Miles are a lesson in devaluation. Heck, I wouldn't count on any of the airlines as we know it necessarily to be around in 30-40 years.