First off, what freaking Republicans are talking about impeachment? Not many if any. This talk of impeachment is, if anything, a democrat attempt to relive the glory days (in their minds at least) of the Clinton impeachment.
Having said that, there is a big difference between lurid sex accusations and, amongst other things, using the IRS to destroy the liberty of the American People.
Do you have that IRS smoking gun that leads to Obama? If so I say impeach. If not you would just be spinning your wheels and increasing his popularity.
Well, is there "a big difference between lurid sex accusations" and lying to a federal grand jury, plus denying a US Citizen their Constitutional Rights (obstruction of justice)?