Yes, when one takes the oath of citizenship, he promises to relinquish his allegiance to other countries. Other countries may still claim the person as a citizen. To hold a US passport, you agree that you won't have a passport from another country.
To have dual citizenship, usually one must be born in one country to parents visiting from another country. For instance, if Arnold's parents were US citizens visiting Austria when he was born (they weren't), or if his Austrian parents were visiting the US when he was born, he could be a dual citizen. Dual citizens are supposed to pick one country when they turn 18, but we don't really enforce that. In fact, we don't bother to stop people from voting in both our elections and foreign elections (like in Mexico), which would have required citizenship in both countries.
Unless someone comes up with more proof that Schwarzenegger claims dual citizenship, it's appears not to be true.