I agree with your reasoning.
Whatever the cause de jour, the Church is in the way. Free love, contraception, abortion, homosexuality, same sex marriage...
To remain truly in the Church is to stand apart from this culture and to oppose it and to be quite unpopular. Some aren’t willing to do this.
We are each responsible for our own little corner of the world. We must talk about the life-giving gifts of Jesus Christ on the Cross and being re-presented in memory of him in the Mass.
We must reach out to others.
Where are most of the Catholics in my town? Probably in the evangelical or Baptist Churches because THEY want to be free of the Church’s rules on these subjects.
We’ve got a big job ahead of us to bring them back. (And we can probably start within our own families — I know I can.)