He is not "infringing" with his words. That comes later. I am surprised he doesn't take a leaf out of FDR's book and threaten or try to "pack" the court. There is no Constitutional reason for the number of Justices to be nine. The Constitution just says one Chief Justice and "other" justices. So that is probably at least three up to however many get appointed and confirmed. That might be "infringing" but not unconstitutional.
Didn't you hear him say he was going to issue executive orders to go around everyone? He really does not care about our laws.