I believe it. I did my first hallucinogen when I was 15 years old. Before that I was a pretty good math student. After that, I became an exceptional math student, and 30 years later still use high level math every day in my job. I don’t believe it created something that wasn’t there, but unlocked some things that allowed me to visualize things in a different way.
Your experience is subjective and non-verifiable and probably not true, but only perceived to be true, but for the sake of argument let’s accept it.
Your experience could be true but is in the vast minority of experience with use of hallucinogens such as mushrooms, ecstasy type drugs, and marijuana.
Generally math skills decrease in those who use these drugs.