Jesus was speaking spiritually. This is typical of the gospel of John, which portrays the spiritual side of Jesus much deeper than do the synoptic gospels. Abraham was known for his faith. Jesus was lambasting the Pharisees for not following in Abraham's spiritual footsteps.
"Jesus was speaking spiritually."
Translation: "He didn't really mean it." Yes, yes. We hear this claim all the time from those who wish to deny the clear words of scripture.