What you are doing here is re-framing the discussion to save face. That is an entirely human response.
Just make a mental note of the outcome and let it inform your comments going forward.
You realize of course that the building of the Capitol Dome - and I assume that's what you are referring to when you say Lincoln built the Capitol and not the entire building itself - was begun in 1854 under Franklin Pierce and by the time Lincoln was inaugurated work had been underway for over five years? And that any slave labor used had likely been contracted long before Lincoln was inaugurated? So your claim that Lincoln himself arranged the slave labor is pretty ridiculous.