see post 54
The idea of the "three black popes" appears to come from modern ignorance that North Africa was in antiquity a part of the Mediterranean world settled originally by the Phoenicians. It's akin to the "Cleopatra was from Africa, therefore she was black" myth. Some web references, bizarrely, even claim that Terence and St. Augustine were black. One might as well say that Abraham Lincoln was obviously a Mexican, because he came from North America.
Therefore, The Iguana is correct: we don't know for certain, but the overwhelming likelihood is that all three were Romans, Berbers, or Punic.
"Africa" was not a continent during the Roman Empire. It was the name of the roman province from the western frontier of Egypt to present day Morocco. Blacks would have come from Nubia via the Nile.
Most precisely, the province of "Africa" was modern-day Tunisia (and essentially what was once the territory of Carthage). Persons referred to in antiquity as "Africans" came from this province. To the east of Africa was Cyrenaica (modern Libya), and to the west was Numidia (modern Algeria) and Tingitana (modern Morocco). You're absolutely right, though, that Romans didn't consider Africa to be our modern continent of the same name.