Free in the sense of freedom as conceived in the original American ideal (what we today would call old fashioned liberalism, which is a race-blind, class-blind meritocracy). "Send" is also figurative in this poem -- the governments in those places would usually find such "refuse" handy to enslave.
“The wretched refuse of your teeming shore...”
Describes my family to a “T”...