You are conflating several issues.
If the person is here illegally, they are breaking the law.
Additionally, people who break the law in this area (crossing the border) typically do not follow it in other areas, so your guy probably has a stolen gun, also illegal.
Nice try though.
It’s a real example.
Many illegals have had their weapons returned to them by U.S. judges after being cut loose by ICE. No crime, no rationale to keep them.
The legal justification is that they may be in the country illegally but that does not strip them of the right to self defense.
It has historical precedent in the Southwest. Mexican vaqueros routinely crossed the border in the 19th century with arms; no one thought anything of it. Mexicans were Mexicans and Anglos were Americans. There was no confusion about who controlled the land, and equally no opposition to ordinary travelers carrying weapons. Everyone did that.
So it’s something to think about: let’s say the entire People’s Liberation Army shows up in San Pedro harbor, in civilian clothing, with their SKS’s. They politely request to come ashore, passports in hand, along with their rifles. Do we admit them? Say sure, you have the right to enter and we have the 2A so just be peaceable, and come on down?
The implications are interesting.