Heck, it's more basic than that. They could explain away existing structures easily enough. New buildings would be a big problem, but we don't necessarily know that there were any. The bigger problem is how the heck they heated the buildings in the winter. The trees and the look of the place is roughly consistent with a PA latitude, but even at MD or VA latitudes heat would be needed in the winter, and they were not further south than that. Where does the wood come from for heat?
Perhaps it is all part of "Those Logic Problems We Do Not Speak Of".
I was just having a little fun earlier. I didn't get caught up in all the problems with the movie's logic. A person can run on end about the illogical aspects of this movie or many other movies for that matter. I chose to see this movie as I saw his other movies- as an allegory. The dilema facing the characters in this film was not where they were going to find wood for heat or how they were going to patch their clothing. Their struggle was with protecting or sacrificing a perceived innocence they sought to create. Signs was a fable about restoring faith, Unbreakable was about balance. Sixth Sense was just about a boy that saw dead people all the time (and maybe a little about justice).
I haven't read many of your other posts, so maybe I'm missing something you meant to point out. Know that I am not trying to convince you to like the movie. I'm just expressing my point of view.