Boatbums makes a good point in her post.
Yes, we are commanded to not grieve the Holy Spirit, but in that same verse, immediately following that statement is the promise that we are sealed by Him until the day of redemption, as if reassuring us that even if we do grieve Him, He will still be with us because we are SEALED by Him until that day.
Ephesians 4:30 And do not grieve the Holy Spirit of God, by whom you were sealed for the day of redemption.
I think that where some people get offtrack on the doctrine of justification by faith is that they think of the "worse case scenario" and try to break the rule by it. When we speak about Scripture's clear teaching about God's grace saving us by through faith alone, the reflex response of those who reject that teaching is, "You mean you can go out and rape and murder and rob banks and you're still going to go to heaven?". Or they will bring up the idea of a person who rejects his faith after he has believed, is this person STILL saved, they will ask. The Apostle Paul had to deal with the same arguments in his epistles because others has come into the churches and tried to sow dissension by insisting that works were required for salvation in opposition to what he had taught them about grace.
The same arguments are still around and it proves that both Satan persists in perverting the gospel and man persists in making heaven the reward for his own efforts - just like every other religion in the world.
But what about the points concerning living as we please in sin or rejection of our faith - can these actions make us lose our salvation? We know from Scripture that it is not by our works that we are saved but by faith in Christ and when we believe in Him and receive Him as Savior, we are born again into the family of God. The Holy Spirit also indwells us and we are given a new nature, a heart of flesh instead of a heart of stone. We are intrinsically changed. If the faith is genuine, then so is the new birth and our lives WILL demonstrate that because we have the indwelling Spirit of God who not only leads us into all truth but who convicts us of sin and our responsibility to live a life pleasing to God who saved us. Some people are more "teachable" than others and, depending upon their past life and natural inclinations, some take longer than others to understand that they are not under the bondage to sin anymore and they are free from its pull. But, to answer the question that invariably gets tossed out, can you live like the devil and still go to heaven, the answer is simple. If you are a new creature in Christ, the draw of the old sin nature loses its power and you will no longer want to continue in sin. A believer will lose his desire to sin and the old ways will cause shame where, in the past, there was no shame, just willfulness - doing whatever you want. If a person says they have become a believer in Christ and they continue in the same sinful ways with no remorse or genuine repentance, then he should reexamine his faith, because real faith causes real change. But it does NOT change the doctrine of justification by faith alone - that is still true and always will be true because that is what GRACE is all about.
Finally, for the one who supposedly "stops" believing in Christ, is he still saved? This answer is a little more complex. First of all, when we are born again through faith, the Holy Spirit takes up residence within our hearts and we have the presence of God always with us. A person who says they no longer believe is either someone who never DID have genuine faith or he is going through a confused time or trial and it must be worked out in his own mind. Would God reject someone who is His child just because he is going through a low rough patch? I don't think so. God is faithful to His promises and He has said he will never leave us or forsake us, he will not cast us out or lose a one of us. God will not reject us if we are truly His. He knows our frame, our weaknesses, He remembers we are but dust and he loves us still. We have the indwelling Holy Spirit - by whom we are SEALED until the day of redemption - and God will not deny Himself, He remains faithful even if ours is weak. Jesus never gave up on his disciples and we know Peter denied he even knew Christ three times. A person who is a believer in Christ is held in God's hands and NO ONE can pluck us from His hands - no even ourselves. He holds us, we aren't holding Him. In time, that person will be able to come through the trial of faith because the Holy Spirit does not give up on us.
There is great comfort and eternal security God ensured would be in His word. The Bible says the things it says for our benefit and we CAN believe it, trust in, base our eternity on it.