No, it only affects the dollar value. You can still dispose of your property (nobody has taken that from you) but probably not for what you think it is worth.
In a free market capitalist society what you are free to do is:
If you think your neighbor is driving the price of your property down, you simply buy his property and then you can do with it as you see fit. Otherwise, you have no right to impose your will on him for your benefit.
Yup. So any way you slice it, my neighbor imposes costs on me. Of course, the fact that his exercise of his property rights might qualify as an imposition of his will on me and his other neighbors, is of no import.
This illustrates the problem with property rights absolutists. The exercise of property rights can and does impose costs on others. The unasked-for imposition of costs is a violation of my property rights.
It's beyond ludicrous to suggest that my neighbors and I have no legitimate claims to recourse if some guy on the block decides, say, to open a biker bar or brothel in his house. Clearly it's within his absolute property rights to do such things. And just as clearly, his actions will affect the rest of us -- not just our property values, but also things like our quality of life, and even safety. He has imposed definite costs.
As is typical with libertarian solutions, those behaviors that are detrimental to the community must be allowed at all costs, whereas community standards cannot be enforced, because it's "legislating morality."
The basic premise of those who advocate absolute property rights is that we can pretend to live in isolation from those around us. In real life, the pretense does not stand up to serious scrutiny.
Many on this thread have staked out a libertarian (i.e., indefensible) position on property rights. Here in the real world, it's better to recognize that there are in fact legitimate constraints to the exercise of property rights, and to lay out a defensible position on how to maximize property rights, and keep the constraints to a minimum.