But "modern" society, as you put it, condemns slavery only because all of us --secular people as well --- are living off of the moral capital of the Judeo-Christian "deposit of faith." To the extent that that deposit dwindles over time, not being renewed by faith, the moral structure becomes vitiated and finally disappears.
As Judeo-Christianity recedes, you've got Islam advancing with its frank endorsement of Koranic slavery which has never been repudiated in the Muslim world; you've got totalitarian slavery in North Korea and allied dictaorships; you've got human trafficking and sex-slavery in the West; and you've got the redefinition of human life as a product and not a person (e.g. through genetic engineering) which provides the technological preconditions for the New Slavery of the Future in which everyone will be a product and not a person.
Without reference at least to the Creationist-Deism of Jefferson ("all men are Created equal.. are endowed by their Creator with certain inalienble rights...") we have no axioms to rely upon which could supply a basis for a theory of human rights.
Slavery has been a given in almost all human societies. In the whole history of global cultures, it's the lack of slavery in our own society is an anomaly.
Why are Christian (and, for a while, post-Christian) societies so exceptional? For the answer, read a bit of Rodney Stark's book For the Glory of God: How Monotheism Led to Reformations, Science, Witch-Hunts, and the End of Slavery or his other fascinating book, The Victory of Reason: How Christianity Led to Freedom, Capitalism, and Western Success.
To be added to my ever-burgeoning reading list.
Cheers!