I think you are arguing against your own case, but I'll leave it at that.
Again, the point of the joke was that there is no “why” to his being black. The point was that she was such a drugged/drunk skank that anybody or anything could have been the father. The punch line of the joke could have as easily have been “just be glad that you aren’t a broom-handle”. Please note also that I am not saying the the characterization of the mother in the joke is accurate.