My knowledge of history isn't detailed enough to know about slavery in the late roman empire, or among early Christian leaders. But if the Pope addressed the issue in the 15th century, it's clear that the Biblical text doesn't explicitly forbid it. My point remains, that religious and moral thought evolves. What is obvious to us was not always obvious.
There may come a time, centuries from now, when many of our everyday activites are viewed with horror, just as we now look back on slavery. How will the future -- where everyone is relatively rich and works more or less as an independent contractor -- judge us with regard to hourly wages? Or the whole idea of having employees? What will they think of the practice of keeping pets? Of eating meat? Of operating slaughterhouses and cold-storage facilities for beef carcasses? Of cutting down trees for paper?
If the time ever comes when we can conveniently do away with these things, we probably will, and then we and our times will be judged (by moralistic twits of the future) as being terribly insensitive to the evils we routinely practice. But now, these things are not viewed by most of us as immoral, and religion doesn't condemn these practices (except for a few vegetarian sects, and maybe Jainism). I think slavery was once like that. We are fortunate that our economic system permits us to do without it.