Congress has already done this at least once before. In 2002, it stripped all federal courts of jurisdiction to hear suits challenging brush clearing in the Black Hills. So far the courts haven't stricken that down.
But looking beyond that, I think the words of Article III are clear in that they're defining the full extent of the judicial power of the United States. But the fact that the power exists doesn't mean it must be exercised, and Congress was given power over whether it shall be exercised.
It wasn't even a foregone conclusion when the Constitution was ratified that there even would be lower federal courts. During the drafting of it, there was a sharp debate between those who wanted there to be no lower federal courts at all, and those who wanted the Constitution to mandate their existence. The compromise that was adopted was for Congress to be given the power to decide the question. Since there was a very distinct possibility that there wouldn't be lower federal courts at all, it's hard to imagine that there was some kind of implied limitation to Congress's power to restrict SCOTUS's appellate jurisdiction, that said that they must transfer it to some lower court that might not even have been created. It seems pretty clear that if such a limitation had been intended, it would have been explicit.
"Since there was a very distinct possibility that there wouldn't be lower federal courts at all, it's hard to imagine that there was some kind of implied limitation to Congress's power to restrict SCOTUS's appellate jurisdiction, that said that they must transfer it to some lower court that might not even have been created."