>>I used to attend a church (Episcopal) where the original pastor had the bright idea of not having a choir.<<
In our Catholic Church we have two Holy Masses on Saturday and six on Sunday. We have a chant choir at the TLM and another group of adults at the 7:30 Sunday morning. The Slovak Mass has a Cantor.
With an organ, NOT a piano, one doesn’t need a choir at all. Our Holy Mass at 6:30pm never has a choir. We sing.
Anyhow, there's so much GOOD music to choose from that changing out the hymns is in my opinion a good idea. That's why so many of them were written in the first place. And yes, we sing "Holy God We Praise Thy Name" as well as many of the other old hymns regularly (that's how I found about the habit of 'swooping' on the line 'infinite thy vast domain'.) We have the St. Gregory and Adoremus Hymnals in the choir loft, plenty of good old Catholic music there.