I think the take away from the NT references is that while they knew Jesus was a local kid born and raised (as opposed to a mysterious stranger appearing out of nowhere), people debated about who he was. IOW, they debated over which prophet he - his neshama/soul - was. The gilgul of John, Elijah, Jeremiah, or some other?
Within that framework, John was the gilgul of Elijahu.
Jesus' meaning is that John came with the mission and message of Elijah-- not that he literally "was" Elijah.
Hebrews (9:27) clarifies this when it says, "It is appointed that human beings die once, and after this the judgment."
Whether there was specultion about reincarnation among the Jews of Jesus' time, I cannot say; perhaps there was. But neither Jesus, nor the writers of the Epistles, nor any of the Fathers of the Church taught reincarnation. This would contradict the reality of the Particular Judgment.