Speaking simplistically, the Spanish and the Portuguese are just as much to blame for American Slavery as the English are.
And let’s not forget the slavery by the Incans, mayans, Aztecs, and Indigenous people of what is now North America, South America, and the Caribbean who practiced slavery in various forms throughout their history prior to European contact. ( not to mention cannibalism and human sacrifice)
I venture to say the Spanish and Portuguese are considerably more to blame for Western Hemisphere slavery if you go by the numbers. Only 5 % of the slaves brought to the New World went to North America, British colonial areas. 95% went to South America and the Caribbean. 09% went to Brazil. When it all started is irrelevant.