Northerners were certainly not boycotting Southern cotton and tobacco.
I tried to see how many opposed the Civil War before it started. Difficult to find.
In “Opposition to the American Civil War” I happened onto the draft riots. Apparently in the North the whites opposed the blacks having jobs wanted by newly arriving whites: (Wikipedia)
The origins of the draft riots come from labor competition. In New York, jobs such as long-shoremen, hod-carriers, brickmakers, whitewashers, coachmen, stablemen, porters, bootblacks, barbers, and waiters were mainly occupied by African Americans. .... The Irish and Germans needed jobs, but they had to compete with African Americans for them. Politicians argued that if Lincoln was elected President and the Union won the war, a large exodus of African Americans would come to the North...Newspaper editors such as James Gordon Bennett of the Herald spoke out “if Lincoln is elected to-day, you will have to compete with the labor of four million emancipated Negroes....the north will be flooded with free Negroes, and the labor of the white man will be depreciated and degraded.” This was the premise for many opposing the draft.”