Well, in many Bibles, the Words of Jesus are written in red, versus black for the rest of the text, so that they stand out.
We take the Words of Jesus as is and understand them unto themselves. Next, we take the words of the Gospels and fit them to the Words of Jesus. Now we have the basic shape or form of the Word of God while Jesus was on Earth.
The rest of the NT should then be read with one eye on the Gospels so that any apparent interpretational conflict is settled not by passing over or ignoring the Gospels, but by examining the words of the men who came and did after Jesus left. Now, we understand the NT as a whole. Jesus said that He came to show us what the OT really was and meant, so that we must look at the OT once we understand the NT and measure the OT by the NT understanding that we have.
I agree that we do have some divergence here. I have attempted to show that portions of the Bible are more important than others - the Sermon on the Mount versus the list of begettors in Genesis, for instance. They are all the Word of God, certainly and must not be discounted.
But I think that for instruction in our daily lives and in the eternal destination of our souls, the Sermon is more immediately relevant than who begot whom.
I don't think any side can be said to take that amount of material "as is". We all interpret some, you through the Church and we through other scripture, and we all take some at face value. So really, the "Words of Jesus" for Apostolics are the words of the Church's interpretation of His words. And understandably, everything follows from there, for good or otherwise.