The chattels were not taken for public use.
The further unpaid forced private use of the chattels was barred.
Somehow, I don't think the fifth amendment was ever intended to allow the government to take, seize or destroy private property, for any reason, and the use to which it was put is irrelevant to the taking of it.
And it is also a matter of opinion as to whether they were taken for public use or not. The Liberals certainly put them to use in securing more power for the liberals in Washington DC.
Additionally, there was no "due process" involved.
"Due Process" at the time would have required a court hearing for every slave being taken by the government. Obviously that was unworkable. It was merely assumed that everyone within a state that had fought against DC, was automatically guilty of "rebellion."
The government simply ignored any law or requirement that it didn't want to comply with.