You're sort of limiting your argument by selecting the Census reports from 1850-1860.
Its the only one I could find. For Southerners to imply that slavery was as widespread in the North as it was in the South is absolutely preposterous.