Actually, we took it by right of conquest. The same way they did. This ancestral possession issue gets very, very dicey, because one's claim is only as good as the last archaeological dig.
Then there is reciprocity. If only blacks can be Africans, then blacks can't be Americans or Europeans by the same logic.
Funny thing,
I’ve known a few American blacks who have visited Africa and have come home to anounce that they are Americans and not African Americans.
Nothing cept American culture there when they get real about it.