The question would be -- was there a precedent or law that would have shown the arrest of Milligan to be CLEARLY outside the law -- when it happened? The answer is NO. That is why the question wound up at the SCOTUS. That is why lots of questions end up at SCOTUS -- because there is a question of interpretation.
Presdent Lincoln -- Gee whiz, President Lincoln was presented with a fait accompli -- Milligan was arrested without his knowledge. He upheld the legality of the arrest, as he might be expected to back his subordiate.
But can his memory be attacked simply for that? No. You just look silly.
Walt