Ask that question to the 120,000 Japanese American citizens sent to concentration camps by an Executive order. The Supreme Court upheld that action.
This is another reason I have such contempt for claiming the Supreme Court is an "ex Cathedra" body. Nope, they are pretty much a political body that will bend with the prevailing wins, depending upon who gets put upon it.
The days of interpreting the law as it was originally intended have long been over, and often time the "law" depends upon which side of the bed one of the Judges woke up on.
As someone said when California Prop 8 went to the Supreme court, "Why don't we save a lot of time and just ask Justice Kennedy what he thinks?"