What they despise is simply that we are not Muslims. There is nothing more to explain about it that is not merely an elaboration on this basic truth.
If all other religions were to disappear from this earth, the muslims would soon be at each other's throats, as is happening now in Baghdad and as has happened in the past throughout the muslim world.