That's a very good point. To the degree Lincoln's actions were unconstitutional, they were still necessary. Notice that the Union "patched up" the Constitution after the war with 3 amendments.
And if people don't think that slavery impacts some people today, my mother was telling me about some of my great aunts, children of slaves in the West Indies. People pass down stories and experiences. It's too bad that a lot of black Americans' ancestors needed Abe Lincoln's "unconstitutional" action to give them what they should have had in the US in the first place.