He's the legal guardian. He doesn't have to give a reason. I just assume it's spite, because denying a parent the ability to be with their child in the days/moments before the child dies is just, well, selfish or something. I understand that he and the Schindlers have an intense dislike for one another, but some things should transcend that natural tendency to be spiteful.
They unjustly accused him of domestic violence and just now Terri's brother just stated he's abandoned her. Maybe its just too difficult for him to rise above accusations such as those.