Excellent point.
I'm not sure, to be honest. In the case of incest, though, I would have to say DEFINITELY not, as there is a potential victim (if a child is conceived, the chances are great that he or she will have severe, life-altering defects).
Overall, though, laws against adultery, fornication, homosexual acts, bestiality, etc. are largely unenforcable and not really deterrents to the acts. I think the key is to try to create the social fabric in which such acts are frowned upon, but criminal? I'm not sure...