The main reason was that some enlightened minds were worried that the Japanese would turn and fight against other American's because they thought they were Japanese first and American's second. Kind of the same way some folks are looking at Muslims in uniform today.
At the same time, there were a number of Japanese American's who served with distinction in the Pacific Theater.