That's true. I went in for Japanese in grad school and had a friend studying Mandarin. He once got embarrassed when he said he wanted to "ask" a question of his (attractive) instructor. But his inflection on the word meant "kiss". She "asked" him to clarify.
But at least each meaning is represented with a different character. So memorize a couple, maybe three, thousand characters and you're good. /s