Nope, you are. Trying to impeach someone who isn't a legitimate office holder.
I'll say it again, merely being sworn in does not a President make. It's takes eligibility(NBC, 35+ y/o, 14 years a US resident) , winning the electoral college (with a majority or being selected by the House from the top 3 vote getters if no one gets a majority), swearing the oath of office, and for the clock to reach noon on January 4th. If any of those are not satisfied, then the dude isn't President.
Again, you put the cart before the horse. Obama IS the legitimate president until he is proven illegitimate. The Constitution provides a method for charging, investigating and removing an illegitimate president and those means must be followed if our Constitution is to be upheld.
The de facto officer doctrine confers validity upon acts performed by a person acting under the color of official title even though it is later discovered that the legality of that person's appointment or election to office is deficient. Norton v. Shelby County, 118 U.S. 425, 440 (1886).
http://www.law.cornell.edu/supct/html/94-431.ZO.html