Perhaps ironically, Gödel's Incompleteness Theorem, which I assume you are referring to, does not really apply. On one hand, we have those things that we can prove with the current set of axioms. On the other hand, we use those proofs to construct a contrary argument premised on non-axiomatic systems to which the Incompleteness Theorem does not apply.
The non-axiomatic nature is the source of the uncanny robustness of the contrary arguments. They do not assert correctness, which no reasonable person can assert (c.f. the Incompleteness Theorem), they merely assert a hypothesis with the highest probability of correctness which is outside the purview of the limitations of axiomatic systems e.g. Gödel.
If you really want to see something different, look into pervasively non-axiomatic computational systems, which have the unique ability to bypass most of the limitations of conventional computing models on conventional substrates. Premising any argument on axiomatic systems places limits that are not strictly necessary.
Are you using specialized terms for "correctness"? If you are, please provide a reference.
Otherwise, (and it may be the late hour), it looks like you are saying you are constructing contrary arguments which need not be correct.
"So if they don't claim to be correct why bother with them?" ;-)
References?
Cheers!