Mostly I am with you.
The direct attack with overwhelming force would in all likelyhood come from a starving band of military regulars, after a failure of supply lines. Other than that possibility, I see little chance that anybody is going to follow anyone that isn’t feeding them well. Groups just don’t hold together without solid logistics.
The roaming singles or pairs will fall to the quiet pip of a .22 short at short range.