Well, that person's lack of PERSONAL RESPONSIBILITY is impinging on your right to do whatever the hell you want!

Until of course, those who rant about PERSONAL RESPONSIBILITY have their own ox gored -- like my friend the engineer who railed against the need to put in ramps and curb cuts, saying it was a stupid, costly, typically liberal imposition on architects and builders...until she had a baby and realized that ramps and curb cuts aren't just useful for wheelchairs, but make the life of stroller-pushing moms a lot easier.

Or a former boss who was a dyed-in-the-wool each-man-for-himself Libertarian....until he had a deaf son, at which point he became pro-ADA pretty damned quickly. No surprise that the same people who hate to accomodate others are pretty quick to appreciate the concepts like "co-operation" pretty damned quickly when something affects them directly.