Put it this way...I wouldn't want to throw a nickel or a quarter as high as I could into the air and then have it land squarely on my head. That would definitely HURT!
We don't get to shoot at low-flying Cessna and Piper propeller aircraft flying above our homes. We also don't shoot at people in hot air balloons who may also be flying overhead and snapping pictures.
At the same time, people certainly DO have the right to quiet enjoyment without opening windows to see drones hovering there with a camera taking high-resolution pics and video. ;-)
I wouldn't try flyin that thang over a danged nudist colony whilst thay wuz shootin skeet!!!
So just don't name yer balloon "skeet!" (or else phhhhht!!!)