Some do, but "most" don't !
Just as the "new" immigrants stuck to the same neighborhoods, until they managed to not only assimilate,but climb the class ladder, many still had enclaves, as they rose, in better sections of the city/state, even other states! They also didn't marry outside of their "group" or religions until fairly recently. Add to that, though there were slave holders in the North East, at one time, there weren't all THAT many of them!
There isn't a single person in my family, on all sides, who has an ancestor who was a slave holder and I can saw the same for most, I have ever known.
You are the one who is wrong and very much so!
Again, you’re just wrong. I’ve done many years worth of research on genealogy, and most Americans are going to have at least one line, if not many, that lead back to places like Tidewater Virginia where slavery was intrinsic to the culture.