Note how he also says "ye have NOT chosen me," showing that their following of Him is predicated on His choice alone to ordain them to everlasting life and fruitfulness. This is a theme throughout the entire Gospel of John, and is consistently applied to all believers, so one cannot say that this only applies to the Apostles alone.
John 6:65 "no man can come unto me, except it were given unto him of my Father."
Joh 6:37 "All that the Father giveth me shall come to me" In instance after instance, God is the active agent in salvation who does the "choosing," and man, who responds, without fail does so and comes to Christ. While the infidels are told that the reason for their faithlessness is because they were not chosen by the Father, are "of this world," and not for any other cause.
Joh 6:64 But there are some of you who do not believe." (For Jesus knew from the beginning who those were who did not believe, and who it was who would betray him.) Joh 6:65 And he said, "This is why I told you that no one can come to me unless it is granted him by the Father."
Thus, choosing to believe is something we do of Free Will; knocking absolute predesitination into the ashpile.
*2 Peter 3:9