The US doesn't recognize the foreign citizenship so, as far as the US is concerned, one is a citizen of only the USA. Of course, foreign countries (like France) are not obliged to follow the US interpretation of this. So, the US says that dual US/French citizens are not dual citizens, but Americans only. France says they are citizens of France too. The US can't really do anything about that, and doesn't. There are plenty of Americans who are dual citizens of the US and Israel, the US and France, the US and Canada, the US and Mexico. It's just that US law doesn't recognize the dual citizenship. Where things become tricky is if security clearances are involved, or someone in power wishes to "get" somebody. Then laws can be read hypercritically in order to bring the power of government to bear on an individual. If I recall correctly, Senator Lieberman is also a citizen of Israel, by Israeli reckoning, even if he's not by US reckoning.
You are correct about the idea of dual citizenship being recognized by other countries.
The US makes it illegal for an American citizen to seek citizenship in a foreign country, vote in a foreign election, or serve in a foreign military.
He's not. Security clearances properly look to the issue of loyalty, since there are not only over 50 countries which recognize dual-citizenship, but security clearances are granted to non-US citizens as well.