But if that man is "yourself" and it's your "latest whim", it's okay?
Why did Jesus assemble the Apostles if He didn't desire unity? Heck, why did He pray for unity in the Garden of Gethsemane if religion is "every man for himself"?
In my church it is a group effort through the deacons and in some cases every member of the entire body of the church has a vote on the direction of the individual church within the boundaries of the Bible and whether or not to accept guidance from an outside association such as the SBC. No one man directs the workings of our church. If our pastor starts preaching on things outside the Word and the deacons/members vote on it he can be dismissed unlike the Pope. No one person should hold that authority over congregates. If the SBC comes out with a doctrine we do not like as a church we can leave the SBC.