I think it wasn't any particular thing that any individual did. I think it was the collection of things to which they finally grew so weary that they just didn't want to deal with it anymore.
They probably saw the election of Lincoln as proof that things would never get better for them so long as they remained in the Union. They were outvoted, and had no ability whatsoever to control their destiny. They were literally a milk cow for the same group of people who control the United States today.
Would they have left if Breckenridge got elected?