Which side was the "liberals" in 1860? The Side that wanted social change and social equity for blacks, or the side that wanted to keep society exactly as it was?
The liberals of 1860 also wanted high taxes, huge spending, protectionism, expansion of government, and many government projects administrated by them.
Kinda like modern liberals.
The liberals were the southern Democrats in 1860, because they believed they had the right to own other people, treat them differently, and exploit them for free labor. That's no different than liberals today pushing for mass invasions of foreigners to exploit them for political reasons, take advantage of a cheap labor force, while discriminating against white U.S. citizens.