My understanding is the skin in the anus (unlike in the vagina) is very thin and prone to tearing easily. Anal sex often time tears the skin open giving HIV easy access.
That, and feces can get into the bloodstream leading to who knows what infections.