It could to an extent be differing perceptions about the same realities - whites are more likely to view an increasing reliance on government handouts and freebies as bad and unsustainable, whereas blacks and Hispanics see it as something free and sustainable. Whites are more likely to fear creeping totalitarianism and a vast expansion of government, blacks and Hispanics to support it.
I think your reasoning is correct; which makes the GOP idea that they can win the majority of Hispanic voters foolhardy.