I totally agree here, and would not be terribly surprised, or really sympathetic, if someone dealt harshly with the fence builder.
His property rights are beyond question, but a common law "eye for an eye" reply, legal or not, is a reasonable expectation.