You're obviously free to say whatever you want if it amuses you. But it is utterly classless to do so given the lack of any real evidence, especially considering that his wife and particularly his children are probably getting these kind of smears thrown in their faces.
You're not helping your candidate, but then, maybe that's not your goal anyway.