They impacted, just not enough. The press refused to pursue the story and Hill's "Stand By Your Man" routine worked like a charm. Once Bill was in office, many American's were willing to turn a blind eye to the allegations in '96, not out of love for Clinton but to protect the Presidency.
If you figure 30% of the electorate is hardcore dem and 30% hardcore pub, that leaves 40% swinging. Perot got 19% leaving Clinton with 12% and Bush1 with 7%. Bush lost by 5% (if memory serves) and that can be attributed to a lackluster campaign, diddling on a weak economy and the press's relentless attacks on "No new taxes" and Dan Quayle as monkey boy. Perot certainly was factor, just as Nader was for Gore in '00.