I think you’re right on this one. I was comparing the barren woman to proselytes, who had to go through Israel in order to approach God. But I think you’re correct..
"For a brief moment I abandoned you, but with deep compassion I will bring you back. In a surge of anger I hid my face from you for a moment, but with everlasting kindness I will have compassion on you," says the LORD your Redeemer.”
God never abandoned any one else to later “bring you back”. It’s clear they are descendents of the Jews and not Gentiles. God never hid his face from the Gentiles for a moment to bring them back.