This, again, makes no sense. If I am "living within the law", then that requires me to respect the rights of "deviants", whoever that may be. You have the right to not have force or fraud, or the threat thereof, initiated upon you. Allowing each individual this right violates nothing. Not killing or jailing someone for selling their body violates no one's rights for several reasons. First, you do not have the right to stop them. Secondly, YOU have the same right, whether YOU excercise it or not. Thirdly, "communities" or groups of people have no more rights than the individual. I know this is contrary to the way you want to get enough people to agree with you, so that it seems ok when the government forcibly harms people in a way you would never do as an individual.
If you allow, or even encourage, pan-handling for example (a non-violent activity), then the hundreds of people who are harrassed by each panhandler...
Well, it may be non-violent, but is this panhandler on private property? Then he/she is trespassing. Public property? Well, thats different(considering there is public property). Then, panhandling must be defined. Is it the simple asking for money, or is it constant begging? And whats the penalty? A fine? Jail? Death?
...have lost a degree of freedom that also translates into more tangable concepts such as lost property values, lost tax base, etc.
Im not sure where this comes from. You do not have a right to "property value", and you surely do not have a right to a "tax base". Taxing me for property I own makes the State the owner and me a tenant.
Your statement of "giving freedoms in a few instances" is based upon erronious, pre-conceived notions that "certain things effect everyone", when that is simply not true. The prostitute that lives in the house five doors down only effects me as much as I want her to. Again, you have to understand PRIVATE PROPERTY RIGHTS!! You control your body and your property - not anyone elses. When anyone comes on your property without your permission, they are initiating force by trespassing. They can't do that. It doesn't matter if its the avon lady, a starving old lady or a doped up idiot looking to steal your lawn mower. The reason for the trespass is not considered. Thats called "The equal and indiscrimitory application of laws protecting against the initiation of force or fraud". No victim and perp, then no crime. This eliminates human nature, favortism and bribary while protecting everyone's rights impartially. Then, we don't have relatives of "the important" not going to jail when caught with "drugs" while the poor guy who has no connections goes to jail. Neither person initiated force, so we would not have a judicial system where some are more equal than others.