Your flawed dichotomy would logically ALSO state that you go to Mass in a barn. Go ahead.
Really?
Growing up, I remember we had one "High Mass" on Sunday, in the Tridentine days, and everything else was dead silence, musically.
Today, only one of our Sunday Masses is without music, the early Sunday Mass.
We use approved hymnals, and much of the music is dreadful.
But, most Catholic music is, and has always been, dreadful.