You are correct, I heard him say the same thing. I remember thinking that it was pretty compelling evidence that HKS wasn't letting any of her family call her. It's not just Virgie saying it. It's Daniel (through the PI) and now ANS's father.
I still think it was mostly Anna (not wanting to talk to family much). When and if Anna wanted to, she would have. I don't quite yet buy that Howard always hung up on Daniel. We've only heard one side of the story, thus far. I'd have to hear much more, from different sources, to make up my mind on the family thing.