I’ve also had trouble with using cotton for face scrubbers. They don’t dry very quickly and they mildew. I’ve also tried cotton acrylic blends or polyester blends, but I’ve found they tend to shed too much for me.
I like the idea of trying a two ply cotton or a cotton thread. I had thought of maybe trying a bamboo cotton blend. Those yarns/threads are just so thin, requiring such small hooks. I usually don’t like working with the small stuff, but since face scrubbers are so small, I’m willing to try it.
Linen, hemp or wool are good for face scrubbers because they exfoliate better and dry faster than cotton yarns.
For thin yarns I just double strand. Wool makes very nice soap saver sacks, also.
Maybe make a face-scrubber sized net, and stick a piece of luffa or other material inside?