Yeah, but if I had to guess, I would say that the BH forms early on and creates a huge gravity well which then causes more matter to be drawn into the area, whch then creates an even bigger gravity well - so on and so on... A small cluster of stars with one really big one that collapses and becomes a BH can then become an even larger gravity well that draws in more and more stars as the eons pass.
That was my thought as well. It would take a massive gravity well or singularity to initiate the clumping of matter from such huge distances. More than could be expected in a more spontaneous and black hole-less event. And since there were plenty of stars at that time anyway, I imagine there were a goodly number of black holes or even neutron stars around after many a hot, fast burning star collapsed.