Look up the definitions of racism and xenophobia.
What you have described in this post is racism. Not xenophobia.
How many foreign workers are now in Japan? In the '70s and '80s, there were very few. Now, Ueno koen looks like a middle east bazaar. Not very xenophobic.
Again, racist? Yes, xenophobic? Not really.
Perhaps some of my examples were a bit fallacious in trying to cite xenophobia. But have you ever met a Japanese person who has spent a considerable amount of time working or studying abroad, yet returned to Japan? They catch all kinds of subtle flack from people. God forbid they were exposed to new ideas overseas. They are sort of shunned and distrusted back home, having somehow become 'less Japanese'.