Apparently, when the cost will be higher than we can bear.
"Find out just what people will submit to, and you have found out the exact amount of injustice and wrong which will be imposed upon them; and these will continue until they are resisted with either words or blows, or with both.
"The limits of tyrants are prescribed by the endurance of those whom they oppress."
--Frederick Douglass