Court packing, while legal under the Constitution, strikes at the necessary independence of the court. As Alexander Hamilton said, "The complete independence of the courts of justice is peculiarly essential in a limited Constitution."
By this point in time, Lincoln had already usurped powers of the legislature by doing things that the Constitution assigned to Congress. He had already ignored a court ruling against him and had imprisoned a judge. He had already trampled on parts of the Constitution (e.g., freedom of the press) that Congress did not have the power to authorize after the fact. He overthrew elected governments of Maryland and Missouri and imprisoned members of the Maryland legislature. His troops interfered with elections in 1862-64.
So, I'm not surprised that Lincoln went after the independence of the judiciary too.
Ah yes, no more unbiased source than that, is there?