Smart thinking.
And I will stand up for the right of worship organizations to keep whatever rules of private association they choose in order to follow their consciences. Whether or not I think their theology is right. Because even if they have made errors, that is not a thing for the state, whose mission is to enforce civil order, to nose into. That is a thing for the church and God to get right between one another.
Maybe an evangelical church would be okay with a fellow like this singing Amazing Grace, hoping that he would get the hint and also yield his own life to the amazing grace of God. But it would be up to the pastor and elders to make that judgment call.