Joseph took his family and fled to Egypt. Both Judea and Egypt were provinces of Rome. They sound like displaced persons. From the Wikipedia entry for displaced person:
“someone who is forced to flee his or her home but who remains within his or her country’s borders. They are often referred to as refugees, although they do not fall within the legal definitions of a refugee.”
https://en.wikipedia.org/wiki/Internally_displaced_person
With regard to being a refuge, the international convention regarding refugees essentially requires somebody to be a member of a persecuted minority group; i.e., has a
“well-founded fear of being persecuted for reasons of race, religion, nationality, membership of a particular social group or political opinion.”
Therefore, Hispanics coming here from countries in which they are part of the majority group are not refugees. Their government is supposed to protect them in their homeland.
IIRC, at this time Judea was not a Roman province, but a client kingdom ruled by the Hasmonean monarch, King Herod. Still part of the Empire, only not officially.