The term cracker refers to the one who cracks the whip, not the edible food stuff which goes nice with spreadable cheeses.
Actually, it refers to neither. Dr. Thomas Sowell and Dr. John McWhorter have both done yeoman's research into various linguistic peculiarities associated with race, and both have some interesting things to say about such terms.
It turns out that "cracker" comes from the root that is also the root of the term "wise-crack". It is an Irish saying (loosely "to crack wise") that was transferred to African American communities post emancipation (since many poor immigrant whites and freed slaves shared the same occupations, especially in the Mid-Atlantic and South). The term actually denotes a loud-mouthed or obnoxious individual, and was used in Ireland long before it was used here. Blacks picked up this term and began to use it to denigrate their Irish (i.e., white) counterparts, from whence it became generalized to whites as a whole.
An interesting side note: what most people consider "ebonics" actually derives partly from a linguistic pattern also borrowed from Irish immigrants, in this case ones who had settled in a suburb of London before continuing (for various unpleasant reasons... or as it would have sounded in their mouths "rais'ns") on to the United States. Language is a funny thing...