Then where do you draw the line? Do we have the moral authority to prohibit alcohol? Tobacco? Cheeseburgers? Bungie jumping? Is your right to cross the street limited by the harm your family will suffer if this risky act results in you being struck by a car and crippled?
Apparently, your answer is to draw no line at all. Anything goes, right? The strong survive; to hell with the rest. I'm just glad I'm not in your family.