Coincidentally I just got a copy of "The Origin of Fruits and Vegetables" today from Amazon.com.
http://www.amazon.com/exec/obidos/tg/detail/-/0789306565/qid=1081900730/sr=1-1/ref=sr_1_1/103-2816519-9044641?v=glance&s=books Just started reading it, but one of the things I've learned so far is that the earliest cultivated fruits and vegetables, not surprisingly, are ones that are very easy to cultivate by transplanting cuttings of stems or roots, like figs, grapes, pomegranates, olives, and dates.
Of course, climate matters, too. Olives are not cold-tolerant, but hate tropical climates even worse than a little cold, so that where they grow essentially defines the Mediterranean climate.
If you are interested in the history of horticulture (I am a sucker for this stuff) you might like this book.
There's also a lot of fascinating information about the origins of agriculture (as well as the origins of writing, animal domestication, shipbuilding, nationstates, etc.) in the great book
"Guns, Germs, and Steel: The fates of human societies", by Jared Diamond. I received this book for Christmas and hugely enjoyed it. It won the Pulitzer Prize, and I can certainly understand why. It's more informative (and a hell of a lot more interesting) than most college courses.