Simple Solution.. MODIFY the zoning, or give the man a variance... The idea that he must match the letter of the laws when he clearly is a special case is nonsense. This is what VARIANCES are for.
When the owner writes a check the “problem” will be resolved.