[1832 Ordinance Nulllifcation cited] - The State of South Carolina very nearly seceded a generation before the war – and slavery was not the issue.
That is probably the bst affirmative argument you could make to that challenge. But in a sense it also begs the question.
Of course South Carolina did not secede, but rather backed down, contrary to their threat, once President Jackson threatened to use force to resolve the crisis. But you are correct that tariffs were the key issue in play in 1832.
I would respond first by saying that the country came a long way from 1832 to 1860. Slavery was not tearing the country apart in 1832 - they thought the issue had been resolved in 1820 and in any case the abolitionist movement had not really gotten any traction yet.
I can't rule out completely that a secession crisis could have erupted for real strictly over tariffs, but it seems much less likely. You could even say that tariffs were an issue in 1860; after all the Morill Act had passed earlier that year but Buchanan had not signed it into law. With Lincoln in charge its signing was a given. Lincoln's stance on protective tariffs was certainly obnoxious to South Carolina and the Deep South.
It's possible in a sense that both sides are right in this debate. In 1860 two very different societies had emerged in the United States, the frontier notwithstanding: an increasingly industrialized, mercantilist North and a largely slave based cash crop agrarian south. The North further was flooded with European immigrants (who sought city factory or mine work or free western land to homestead on), whereas the South remained largely static. If we say that economic causes were central to the war there is a great deal of truth in that; southerners felt their livelihoods were at stake. But it is also worth bearing in mind what the central prop of that economy was: chattel slavery. Without that institution and the nearly 3 million African slaves needed to sustain it, the overwhelming dominance of cash crops like cotton would have been hard to sustain.
All the vital roads crossed in slavery. Slavery was the basis for the South's economy. Slavery was the basis for the South's caste-based society, where even poor southrn whites could feel part of the elite in some sense. Slavery was the basis of enormous abolitionist agitation in the North in the 1850's. And slavery, in the end, proved to be a catalyzing issue that united the South behind South Carolina in the crisis of 1860-1861 in a way in which tariffs did not in 1832 or any other time.
So back to the question: "Can you imagine the war - or secession - happening if slavery had not existed?" If the South was strictly free-soil, they would have ended up with a considerably different society than they in fact had - one much closer to the North. And without the dominance of cash crops which discouraged immigration and development of industry, they also would have had less opposition to protective tariffs. No slavery = no opposition to tariffs. Because slavery was the thing which more than any other factor made the South the distinctive society that it was - the society which was so different in its economic interests from the North.
Allow me to suggest an alternative interpretation:
No States rights = no secession
Without the concept of States rights promulgated by Thomas Jefferson and James Madison, there would have been no mention of seceding from the Union, whether the issue was federal violation of the Bill of Rights (Kentucky and Virginia Resolutions, 1798), war with Britain (the Hartford Convention, 1814), the unconstitutional expenditure of federal funds (Mr. Jefferson's Declaration of 1825), tariffs (1832), or slavery (1850s-1860s).
The States rights ideal of the Jeffersonian republicans was the 'root of secession' - and a most admirable root it is, indeed...
;>)