The judicial power of the United States, shall be vested in one Supreme Court, and in such inferior courts as the Congress may from time to time ordain and establish. The judges, both of the Supreme and inferior courts, shall hold their offices during good behaviour, and shall, at stated times, receive for their services, a compensation, which shall not be diminished during their continuance in office.
This portion clearly establishes that, apart from the Supreme Court, Congress is at liberty to determine the structure of the federal court system. The other limiting condition on Congress for maintaining the structure determined by them is the good behavior clause which only allows them to maintain their continuance in the office. It does not limit Congress as to whether that office exists or not except for the Supreme Court. It must exist. If, for instance, the Congress wished to make the organization of the federal courts completely state based, I see no reason that laws could not be changed or written by Congress to achieve that type of organization. Thus it appears that Congress could by means of organization reduce the offices available to be filled. That is due to the fact that there is no number associated with the federal judicial system in the Constitution. And that the determination of the structure of that system is vested entirely in Congress.
To constitute tribunals inferior to the Supreme Court;
Well, yes - there's no required number of judicial officers that must exist, either as a minimum or as a maximum. But you really can't get rid of a sitting judge simply by reorganizing in such a way as to eliminate his position. Otherwise, Congress would never need to bother with impeachment of judges - they'd be able to do an end-run around the impeachment process any time they liked, simply by reducing the number of district courts by one and selecting the sacrificial lamb to be downsized out of a job. As I said, though, the courts could be reorganized to eliminate posts that are currently vacant anyway.