Many countries permit dual citizenship. The US is one of the few countries that specifically prohibits its citizens from holding citizenship in a foreign country, voting in a foreign election, or serving in a foreign army.
The US doesn't recognize the foreign citizenship so, as far as the US is concerned, one is a citizen of only the USA. Of course, foreign countries (like France) are not obliged to follow the US interpretation of this. So, the US says that dual US/French citizens are not dual citizens, but Americans only. France says they are citizens of France too. The US can't really do anything about that, and doesn't. There are plenty of Americans who are dual citizens of the US and Israel, the US and France, the US and Canada, the US and Mexico. It's just that US law doesn't recognize the dual citizenship. Where things become tricky is if security clearances are involved, or someone in power wishes to "get" somebody. Then laws can be read hypercritically in order to bring the power of government to bear on an individual. If I recall correctly, Senator Lieberman is also a citizen of Israel, by Israeli reckoning, even if he's not by US reckoning.