To the contrary. It is not a problem to harmonize the two at all. And it doesn't involve throwing out Paul unless you can tweak just a tiny bit of support to spin your way to support Catholicism.
"by works a man is justified; and not by faith only" (James 2). Direct negation of "Faith alone".