I've talked to several well-connected folks at the remaining clothing factories to see what stands between them and reverting to US production. They all say the same thing: at the current price point and volume of mass-market clothing they could not afford to pay a US workforce, even with extensive automation. Moreover, the work force a the boutique plant is all Latino (read illegal), so they are really not on the same playing field. It is really frustrating. Unless you sell a product that you can make quickly in high volume with a small work force and can sell for many times what it costs to make it, you can't make it in the USA.
I talked to a guy years ago who was a buyer of men’s clothing for Sears. He said that Sears bought a dress shirt for $2 and sold them for $30. I have no idea what the overhead was for the store at the mall. A lot, I’ll bet.