Yes, that is what Islam teaches but what do the people think?
Read a little about what goes on in Iran concerning the Jewish and Christian minorities.
Since the mullahs have been in control of Iran for more than 30 years, it tends to disprove your claim that Muslims want to kill all non-believers, since if this were true presumably there would be no infidel minorities left alive there.