Actually, they did die physically, just not right then. There's no telling how much time passed from the creation until Adam/Eve ate the fruit. But at that instant, they transformed from immortal to mortal.
Sin brings spiritual death and that is what happened to Adam and Eve.