I think it's Japan's proper role to limit its neighbors' adventurism, so I support arming Japan to the hilt, xenophobia or no. Between the US, China, and Russia, any renewed Japanese ambitions of empire are similarly constrained.
I think any person has the right to self defense in any fashion they see fit. However, nations do not...they are armed at the superpower's sufferance, and its convenience. I think it inconvenient to extend the right of nuclear self-defense to nations that have declared war upon other nations in the last century by sneak attack or by civilian-killing terrorist stalking horses. Japan falls solidly into the the former camp. And if you expect the rearmed Japanese to be controlled by other regional powers, you are simply building up another enemy to be figured in. I don't think that's good long-run strategy, advancing the military might of a people who think they're culturally superior to others, if you intend to STOP wars. Maybe if you want to START them, sure.