Read Judges. The Israelites broke the promise by not killing all the Canaanites, and God voided the contract.
The Jew was never "the apple of Gods eye". The Israelites maybe were. But while all Jews are Israelites, only a small minority of Israelites were ever Jews.