At least one of the natives joined Jews in the battle against the corrupt Kings. Rahab the harlot.
It is not unbiblical to conjecture that other natives did as well.
Unfortunately, if you take the Bible's word literally, with very few exceptions the Canaanite natives weren't allowed to switch sides. They were to be exterminated or expelled.
This contrasts dramatically with the account of the exodus from Egypt, in which a "vast mixed company" leaves Egypt with the Israelites. 40 years later there is no mention of a mixed company when they enter Canaan, with the obvious implication that they were assimilated.
Was Caleb an Israelite? At Numbers 32.12 he is called a Kenezite. Elsewhere he is treated as belonging to the tribe of Judah. Heber the Kenite (Judges 4.11 and 5.24) also seems to be a non-Israelite.