The opposite problem is actually the case: how did some black holes (supermassive and beyond) grow so large? Based on the matter density of the universe and calculating even from the earliest possible starting point (shortly after star formation began in our universe), you don’t get anything remotely close to supermassive black hole size. In fact, what you find when you run those numbers are the largest of the stellar black holes. So those black holes formed as we understand it today and grew over time to their size today.
But there’s an enormous gap in size where we don’t see many black holes. In fact, there’s debate about whether any of the objects in the “intermediate” class are actually black holes. If the largest ones grew naturally, there should be some of just about every size from the smallest to the largest. Instead, we’ve got small and super-sized.
Thus the need for ‘Dark Matter’...................