I think the vanishing of 15 to 20 million would impact the housing markets a bit, so builders/roofers/dry-wall guys may have a rough go of it for a while, as there are already more homes in some areas than buyers, but it would recover.
I do not think it would create much of an impact since those 20 million illegals live in only 150 houses.
The economic fallacy of illegal immigration is that the primary impact of illegals is cheap wages.
In reality, the net economic impact is the economic activity of 20+ million added consumers to the United States who are propping up the system to keep the US economy from collapsing in on itself.
If they all left, our economy would implode and the United states would be pushed into a depression that makes the 1930 look like nothing.