I think rejection of caste helped Islam become entrenched in Bengal, and is a factor in the spread of Christianity, which is especially strong among the Dalit or “untouchable” caste.
I think you are correct. It would be hard to live as a Dalit or in a caste society generally.
Caste is very much alive among Muslims and Christians in India. Its a social system not religious.