If I was a neighbor and they were quiet and didn’t block the street with parked cars, I wouldn’t care. However, if you look at 2343 Miramar Blvd in University Heights, OH, his “house” is nothing like the others. It’s obviously been modified to act as a place where many people can regularly gather. I can see the neighbors’ and city’s point here.
He probably has a large family. Orthodox Jews often have a lot of kids so that may be to house a large family.