I believe this is factually incorrect. Pythagoras had all sorts of ideas about the relationship between music, the universe and higher thought.
He died around 500 BC, about 1000 years before Augustine.
You’re right in a way. It was not necessarily Pythagoras, but one of the thinkers “of the school of Puthagoras”—that is, people he hung with, colleagues and students—who were considering these issues around 500 BC. Lacking any concept of science as we understand it, they looked for patterns in the observable world as a guide to the principles by which the universe was made. They observed the way a harp strong produces sound as its length is divided, and from there extrapolated our ability to perceive the resultant tones to a general perception of beauty. But this is a far cry from the higher math developed by Liebnitz and Newton and their successors.