“If you insist on violating his rights, he has the right to defend himself using deadly force, and I hope he does.”
And if he insists on violating my rights, I have the right to defend myself using deadly force (but I would add “if necessary”).
What’s in dispute is where the property owner’s rights end and the rights of anyone else begin.
That's the point: if his rights are properly understood, there is no limitation to them. He doesn't have the right to initiate aggression, and therefore he can't molest you, steal your stuff, vandalise or what have you. If his actions affect nobody's person or property except his own, then anything goes. Or, if all affected parties are consenting, then again anything goes. Those are exactly his rights, and there are no exceptions of any kind.
The only one who can impose limitations on a man's rights is the man himself. He does that by making contracts, whether express or implied. If he marries, he's making a contract whereby his spouse can live in his house without being called a trespasser. If he takes a job, he's making a contract whereby another person can suddenly tell him when to get up in the morning, or what to wear, or whatever else the contract might entail. In our dealings with others we make implicit contracts constantly. The McDonalds, by displaying an "open" sign, offers a contract that we can enter and eat unmolested. By entering, we accept the contract, and in return contract that we'll line up, pay for our food, eat it without bothering the other patrons, and bus our own trays.
What’s in dispute is where the property owner’s rights end and the rights of anyone else begin.
All of our rights begin and end with property and contract. We own ourselves and our stuff, and we can do anything we want to our selves and our stuff, but we can't do anything whatsoever to anyone else's things or stuff without their permission.