Sorry, but the tolerance for socialism, marxism, and wife-swapping arises in the great majority out of a desire to be selfless, and out of a impaired sense of self.
For some — the bullies of life — it is selfish, sure. But for the majority who get caught in them, it is a lack of strong sense of self that allows them to become trapped by the trappers of men.
I think you are confusing selfishness with strength of will.
They are separate things.