Your whole premise, and the reason why I think that Luther was wrong, opens a pandora's box for a cafeteria style canon. i.e. one guy accepts all while another denies James, Jude and Hebrews.
This situation of cafetria style is what we are saddled with, whether we like it or not. God has not told us which books belong and which don't. The group that put the origional so-called NT together was not necessarily inspired. Well, I just thought of something. Perhaps it was inspired, but how do we decide whether it was or not? We have to examine the books one by one, just like they did, and we are thus unable to escape the cafeteria. In fact, I don't see how you can say the cafeteria is bad, granting that the origional assemblers of the NT were forced to go to that exact same cafeteria.
These are either the word of God, carrying all the weight that implies, or they are not. This is not just simple logic but basic to the faith.
I agree with you on individual books. Cutting out certain verses and not others is irrational and destroys faith. However, I am sure that one can (and must) make descisions at the book by book level: for example, War and Peace is NOT scripture.
By your standards, the same thing could be said about the Letter to Philemon!
Philemon was written by Paul, and hence, by virtue of being written by Paul, it is golden. So, no, by my standards it clearly makes the cut, since "everything written by Paul" is one of my standards. :)
BTW, how do you know the Apostle John wrote the gospel with his name on it? Or for that matter, how do you know Mark or Matthew wrote the gospels with their names on them. Who are you trusting to get this information?
I trust that the church has correctly attributed these works to the correct authors, from the earliest times to the present day. Moreover, skeptical researchers have tried very hard to debunk them, and have failed. This makes the attributions very strong.
What do you think of Bishop Papias?