What's the situation with mom? Seriously unfit? Less fit than you? A basically good person? Someone to protect the kid from?
Personally, I think it's enough that she moved some random guy she met on the internet into her home. That speaks volumes about the judgment of his ex, especially if they have daughters.