Article 4, Section 2, Clause 3 of the Constitution places the subject of slavery within the purview of the States, not the federal government.
This concept is reiterated in the historical records.
{Sorry for the need to cut & paste - the Library of Congress disables a link]
Mr. HUGH L. WHITE. When the Constitution was framed, the great and leading interests of the whole country were considered, and, in the spirit of liberality and compromise, were adjusted and settled. They were settled upon principles that ought to remain undisturbed so long as the Constitution lasts, which I hope will be forever; for although liberty may be preferable to the Union, yet I think the Union is indispensable to the security of liberty. At the formation of the Constitution, slavery existed in many of the states; it was one of the prominent interests that was then settled. It, in all its domestic bearings, was left exclusively to the respective states to do with as they might think best, without any interference on the part of the federal government.
http://memory.loc.gov/cgi-bin/ampage?collId=lled&fileName=004/lled004.db&recNum=607&itemLink=D?hlaw:1:./temp/~ammem_uMGu::%230040608&linkText=1
---
Journal of the Senate of the United States of America, 1789-1873 / SATURDAY, March 2, 1861.
Article 6. No future amendment of the Constitution shall affect the five preceding articles, nor the third paragraph of the second section of the first article of the Constitution, nor the third paragraph of the second section of the fourth article of said Constitution, and no amendment shall be made to the Constitution which will authorize or give to Congress any power to abolish or interfere with slavery in any of the States by whose laws it is or may be allowed or permitted.
http://memory.loc.gov/cgi-bin/ampage?collId=llsj&fileName=052/llsj052.db&recNum=378&itemLink=r?ammem/hlaw:@field(DOCID+@lit(sj05267))%230520379&linkText=1
----------
So, no - the superficial cause was slavery.
The underlying cause was the federal government (along with no too few of the other States) were abridging the Constitution, and eroding the oh-so-carefully drawn lines between federal and each sovereign State's authority.
Article 4, section 2. They absolutely refused to enforce it.
It may be an immoral clause, but the legal system is supposed to enforce the law as written, even when they don't like it.