The common defense. That is the key.
People are advocating torture in order to prevent individual terrorist acts. While these acts may be brutal and horrible, they do not threaten our common existance as a country.
People are advocating extending the common defence to the point where we are protecting each individual and preventing each individual act of violence. This is a whole different matter, and requires a much heavier hand of government.
Great mischief has been done by various governments in the name of protecting the innocent.