What you ignore is Dred Scott, which held Lincoln's position of non-expansion to be invalid. Lincoln's desire for a lily-white west, free from blacks - SLAVE OR FREE - was part and parcel of his economic "free-soil" policies, and his separtist white-supremacist attitudes.
Dredd Scott dismayed Lincoln and the republicans.
But there was no doubt that the Congress had the power to legislate for the territories. That is what the slavers knew, and that is why they tried to duck out on the Constitution.
As for the other, Lincoln is amply on the record well before the war saying that the D of I was meant to apply to black as well as white.
That would make your statement that he preferred a lily white west hard to show in the record.
It is actually just a construct by you that will not bear even the least scrutiny.
Walt