I view math as something much deeper then language. Languages come and go, there are no ‘linguistic truths’ that are say comparable to theorems (e.g., Fermat’s Last Theorem). Language may not even distinguish us from animals. Clearly there are examples of animals communicating, sharing information and there is structure in its make up. This discussion could go on for days but for myself I think there is more evidence for the Platonist interpretation. (I prefer to call it the Pythagorean interpretation! Why you may ask? Remember the Disney movie, ‘Donald Duck in Mathemagic Land’, the portrayal of Pythagoras & pals? I loved that! Those guys still give me a chuckle! I used to have a screen saver of that scene!)
I hear you, but Fermat’s Last Theorem isn’t the thing that lasts, it’s the truth that Fermat’s Last Theorem proves.
Language can (could) convey the same truth, although we tend to convey more than logic in our language so it’s much less precise.