Indeed. And if the South had been a fraction as successful at attracting immigrants as the North, they would have had many immigrants serving in Gray. It's the story of America, as you said.
Very few immigrants were interested in moving to the South, where they would have to compete with slave labor and be treated like white trash by the plantation lords.