Absolutely. Assumming of course, that such costs, are indeed paid by the rest of us. That's where the unfairness comes in. If we shipped all the meat-packing, repetitive tool-and-dye, sewing, parts assembly, etc. overseas, we would not have a social cost considering that these jobs are not filled by Americans anyway. Enforcing non-economically viable standards is always a lose.
We most assuredly would have a social cost. Once upon a time those were the jobs in what today are underclass ghettos. And once upon a time those underclass ghettos had stable working class communities. Those light industrial, stable jobs were the ladder by which Irish and Italians had climbed out of the ghetto and they were eliminated at precisely the time Blacks were going to follow. The elimination of many of those jobs, and their being filled by illegals had a devastating impact upon urban blacks. Without stable, light industrial jobs there is no way out of the ghetto.
I think a permanent underclass is a massive social cost.