If that were to happen...Joe Biden would be immediately sworn-in and he then appoints a VP that must be confirmed by the Senate.
Amendment 25
2. Whenever there is a vacancy in the office of the Vice President, the President shall nominate a Vice President who shall take office upon confirmation by a majority vote of both Houses of Congress.
Taking the oath is just one more requirement to be President. The Constitution merely requires it be done before entering into the execution of the office. It does not, by itself, "make" someone President. The Clock and Calender do that, unless the other requirements are not met at that time on that date. There is no requirement that any particular person administer the oath, or even that it be taken orally. It's generally been the Chief Justice, but not always. It's usually not the Chief Justice, or any Justice, that does it when a VP takes over because the President has died, or in the one case resigned. Only two have been impeached. Andrew Johnson, who took over when Lincoln was assassinated, and William Jefferson Clinton. Neither was removed from office.