It was the agrarian society in which slavery thrived in the South, don't forget.
Your point about slavery in the agrarian south is a valid one, but the North wasn't free of slaves because it was industrialized -- it was free of slaves simply because it didn't need them. By the time the North had become heavily industrialized in the 19th century, they were able to rely on Irish immigrants to fill many of the roles that were not much better than that of a slave (especially in the mining industry).