The authority of the Church is not from men. Sticking to your own doctrine, however, is.
It IS from men. Jesus never told me that I would go to hell if I refused to abstain from meat on Friday and then didn’t participate in a manmade ritual of absolution.