Then don’t give them US citizenship.
Sure, it’d be easy enough to ask naturalized citizens to renounce the old county as a condition of gaining U.S. citizenship. But with the respect to Israel, under their “right of return” law, they hand out Israeli passports to any Jew in the world who wants one. So, in the case at hand, we’re likely talking about mostly birthright American citizens who later acquired Israeli citizenship by virtue of being Jewish. I can’t see a realistic way for the U.S. to easily stop that sort of thing even if we wanted to.