The Courts are organized by Congress and the current size of the Court, nine members, was set in the 1860s, it has had less than that, by law, at times.
Since FDR did it, it cannot be much of a law ...