IMO it would be far better for Japan to not allow many foreigners in - particularly ones incompatible with living in a civilized society. Yes, Japan’s population would decline. So what? They’ve got 120 million people crammed into a land area about the size of Montana. If it became less crowded, it could easily improve the quality of life. Real estate would become much more affordable for example.
Real Estate is already reasonable. I have seen houses for sale on large lots for well under a 100K. One house was 150,000 USD and it was a fantastic house beautiful garden, roomy and modern but old. You could not touch that house even where I am for under 400K USD. Japanese owned homes, passed them down to their children who also owned homes when they died. Property taxes and maintenance was too much and homes were just abandoned. I saw homes offered for FREE, just have to repair. Could not tear them down and rebuild. Some were in okay shape most not. Japanese friend told me they are free but expensive since many of the craftsman need to rebuild long since retired.
Population of Japan in the 1940s was about 75 million. That’s more than enough people. Population decline would mean the average Japanese family could have a living space larger than a single car garage.