The pastor definitely cannot be removed by the SBC. Each church is wholly autonomous. Whether the church can be expelled, though, I am less certain. I used to attend an SBC church and the pastor retired and was replaced by a woman (not permitted in the SBC). We left the church, and shortly thereafter, the church was no longer a part of the SBC. I’m not sure whether it was kicked out or withdrew, though.
In many cases if a church withdraws from the SBC, it has to make a large payment to do so.