Not if such swearing occurred under demonstrably false conditions, in which case Plugs the Wonder Gaffe would become President. Same thing if Joe Jones impersonated Senator-elect John Smith at the swearing in; the swearing is void on the spot (or, once the personation has been demonstrated), and the lawfully elected person, Smith in this example, would be sworn in and take his seat.
.... and Obama may have taken a perjured oath!
When it is proven that Obama is foreign born he will removed from office. Until then he is our sworn President. The same as in the hypothetical senatorial case you presented - as you mentioned, the imposter would be removed and the rightful senator would be sworn in.