One way it could be done would be if the court declares the replacement of the Judiciary Act as unconstitutional — say, because it considers packing the court would put too much of a burden on its duties under Article III of the Constitution. Who'd be in a position to say otherwise? After all, the law against abortion was declared unconstitutional on privacy grounds, even though the Constitution says nothing about privacy,
That would be judicial activism just as much as Roe.