As long as you’re within the towns building codes you should be fine. Believe it or not the gov on the local scale is usually very reasonable and will toss aside idiotic complaints.
I hope you are right. We have local people doing all the plans, etc. Everything has been a great experience except when 2 neighbors showed up yelling at the commission. Not so sure I want to live there now.....