It's not the sex that's the problem. It's the people who either don't pay the gift tax, or don't claim the income that becomes problematic.
Plus, if you're taking funds out of a organization that enjoys tax-exempt status, and you're using those funds for your own personal benefit or entertainment, that too is a problem. The Feds are right and obligated to investigate both allegations. If they're going to make me pay my full tax liability, they damn sure better be making Tiger Woods - and his whores - pay theirs.