If there’s enough mass there to imply a large-enough Swartchild radius, even if it was dark matter, I think you’d still end up with a black hole.
I’ve wondered if there is some sort of association of dark matter to the material that “disappears” into black holes?
True. The dark matter hypothesized here is fermionic, so there is degeneracy pressure supporting the clump just like a neutron star or white dwarf. As in those astrophysical objects, if the clump becomes large enough, it will necessarily collapse to form a black hole.
IMO, to be honest, there is probably a lot of matter that failed to form anything. I think that is all dark matter is.