Not true, local government has been enforcing set-back rules on your property for many, many years.
Not after the house was built and approved e.g., inspected and given a CoO. I know better.