I agree. I know flakes of Christian and Jewish background who are into sufism and I think you’ll find an increasing number of movie stars and other celebrity idiots taking it up. To them, it’s Unitarianism with finger cymbals (although orthodox sufis aren’t really any much different from other Muslims - but the celebrities’ mental image of sufism is based on the charmingly ethnic whirling dervish branches).
However, I don’t think they will be able to drop the rest of Islam, and in fact, I don’t think they really want to. Actually, if these modern Sufis posed any threat to main-line Islam, it would be attacking them...and it’s not.
Sufi islam is just as militant as the rest of them. Al fuqra (sheik Gilani) is sufi, just as an example.