My added word.
I think he was within his property rights, but I think the way he is doing this makes him a moral slob. Which is not to say that people who own the homes are angels or even innocent victims. They are guilty of ignorance, perhaps stupidity, and if they try to use force to "fix" the situation, moral slobs themselves. IMO
It's fine to be unhappy about your own stupidity but to break the law in response to your own stupidity takes stupidity to a new level.