The pejorative meaning of "cult" has connotations of (usually sinister) mind control and total reliance on a leader. There is often isolation from nonmembers. I'd say the Amish et al meet the isolation standard, but not the devotion to the leader one.
It has been argued that the Branch Davidians are a Christian cult, but I think a lot of that was simply propaganda to try to make the massacre look better. (EG, there is no real evidence that Koresh had relations with underage girls; the sensational congressional testimony from one girl appears to have been instigated by her father, who blamed Koresh for the failure of his marriage)
The Jehovah's Witnesses come close to the negative definition of a cult, IMO.
None of these groups are as controlling as the Scientologists, Hare Krishnas, Children of God, and so forth.
The begging off that people can't maintain that a group is a cult is ludicrous. This is like saying that one can't maintain over time that a union is a union or that a Mob family is a criminal enterprise.
Groups are cults whether one has the will to face the facts or not. The brainwashing and indoctrination that goes along with the cults along with the controlling natures often leave their members confused, angry and disillusioned. They keep their members in line in a lot of thuggish ways that aren't necessarily always obvious. They stand to lose everything - family, friends, associations, even their liveliehoods if they leave the cult. There are a lot of ex-Catholics that can vouch for this just as there are many ex-Mormons and Jehovah's Witnesses that can say the same.
But there is no mob in America.. lol.