I can't think of how to distinguish discovery from invention in mathematics, but there is the problem that mathematical systems cannot prove their own truth. That seems a strike against the assertion that mathematics is God's alphabet.
That's reasonable. Truth is always "useful," for it is always more efficient than a lie (and lies can get you in trouble). Heaven knows Revlon, et al., does well because women widely believe that there is practical "use" to be had from projecting an image of youth and beauty (helps ya get a husband, or even certain kinds of jobs, etc.). A composer knows how to "use" harmony to express what he wants to convey in a musical composition. Etc.
We could say these are examples of the "immanent" uses to which truth, beauty, and harmony can be put. But it seems to me their "usefulness" depends on their universality, which suggests to me that they are transcendent values. That is, they are beyond the reach of proof.
Similarly, mathematical systems, as you note, cannot prove their own truth. Yet mathematics is "useful" to us all the same; and its utility may consist only in the simple fact that mathematics is "God's language" (so to speak). But then we can't prove the "truth" of God either, in the sense of "proving" that He "exists".
We seem to have some categorical problems here....