Most men will be jealous. Most women won't care. I sure don't!
Compared to the checkered past of Rudy G., the odd problems of McCain and his wife, Newt's Divorce, the Hillary and Bill fiasco I think that is a pretty mild issue. Pre-Clinton it might have been more of an issue, but after the train wreck of 24 hour news coverage and the Clinton years, people's shock meter is reset IMO.
Her personality might be a bigger factor, and I have no idea bout that. Is SHE conservative or a Hollywood trophy wife?