Previous owners of all three aforementioned territories invaded Israel with the intent of wiping it off the map and expelling or exterminating all Jews. They lost.
Israel allows them and their descendants to remain out of the goodness of their hearts. Until such time, if any, as they make a peace treaty with them as Israel did with the Sinai Peninsula, they have no RIGHT to remain, only a courtesy to remain on good behavior as Israel sees fit to allow.
Amen.
To the victor go the spoils. And in the case of Israel which has not been the aggressor - think Six Days War and Yom Kipper war - this goes doubly so.