Moe said that he and Stern asked Tas to stay with Anna. So it's not like he was deserting her. There was someone there.
But Anna could've fought off Taz if she had to and escape. See i'm working under the supposition that this constant 24/7 vigilance around Anna was more captivity than protection.
Moe *and* Stern asked Tas to stay? See, that doesn't fit, either.