"In all Cases affecting Ambassadors, other public Ministers and Consuls, and those in which a State shall be Party, the supreme Court shall have original Jurisdiction. In all the other Cases before mentioned, the supreme Court shall have appellate Jurisdiction, both as to Law and Fact, with such Exceptions, and under such Regulations as the Congress shall make.’
I'm not a lawyer but that looks like Congress does have some significant authority over the court.
You are correct, you are not a lawyer. Look very closely at this sentence. It only speaks to to the issue of jurisdiction, appellate or original.