It's the same as calling someone a racist. When the Left becomes too comfortable tossing out the racist card, it eventually becomes obvious that the charge is only being made for political purposes.
I think that calling Limbaugh a birther would be accusing the person-too-far, and the charge of birtherism will begin to lose its value.
-PJ