Look at the context of the sentence.
He is discussing believers. They are a separate group from those who would take them away - this other group He refers to are the “no one’s”.
What He is saying is that if you are a believer there is nobody else who has the ability to make you give up your faith. He doesn’t say that it is impossible for YOU to reject your faith. However He does make the promise elsewhere that if you are His He will finish the good work He started in you.
As we have free will still, we are still able to turn our own backs on God if we become lax and grow further and further distant from Him. It’s not Him taking it away from us, it would be that own person’s decision to say No.
Why on earth they would ever do so is beyond me. But we have seen great people who started out strong, were by all accounts faithful, some even great preachers, by their own words bringing others to Christ, only to later in life fall away.
I just take God at His Word and that He has my back, He wants me with Him, He’s looking out for me, wants the best for me, and will complete the good work in me, despite me, for the sake of what Jesus went through for me and everyone else.
Now if you are a believer and you stop believing, for any number of possibilities, you don’t believe that God has your back, you may stop believing in all of it. People have free will and in this life God does not make it impossible for you to reject Him, before or after you are saved, He doesn’t alter your free will so that it is impossible once saved, to not say no to Him. Mature belivers would find such an act crazy to even think about doing, but it isn’t out of the realm of possibility.
Keep in mind that “free will” concept also includes the idea that man is responsible for his salvation. He can then take credit for being one of the saved. A dangerous road to walk.