If all the illegals were to be properly accounted for, the answer to your question is a resounding YES.
If you don't have former third world residents living 16 to a swelling, then you aren't living in a large town.
I don't know about the rest of the US, but where I live, residential overcrowding is a problem and several local jurisdictions have either passed laws to deal with the problem (by restricting the number of people that may live in one residential unit) or are considering doing so.
Most of the neighborhoods so affected are older ones where the houses are less expensive.