The curse of Ham, was a curse on Canaan not Ham (who had three other sons)
God foresaw what that line of people would do to resist Israel, hence the prophecy placed on them.
Not unlike that of Jacob and Esau (referring to two nations), Gen.25 cf Rom.9
More important, Noah could not CURSE what God had already BLESSED. (Who wins that rouhd?)