Frankly IMHO even that would be a stretch.
There often is time after a meeting concludes for interested persons to come up and talk further with the speaker. I frankly find it incredible that a speaker would be so impressed with the brilliance of a question that they made it a point to try to have someone seek out that person (who happens to be of the opposite sex) to invite them to dinner.
We must not allow our support for any candidate cause us to deny what we know to be right and wrong.
That's the bottom line. IF the charges were proven to be true (again, I hope they are not true), would that affect your view of the man? Or are you only concerned about the sins of your opponents, while your guy gets a pass?