States Rights had been an issue dating back to the very ratification of the Constitution. Slavery was a very small part of a very complicated issue (that still is being debated today).
I think that we can agree that Slavery was a smaller issue at the time of the founding of the country, but by the 1840’s it was inextricably bound up with so many issues that it’s hard to see how it could have been dealt with by the political system as it then stood.