Ah yes, a favorite retort of anti-Catholics; however, this verse clearly demonstrates that this is not true (added emphasis in the verse is mine):
And when there had been much disputing, Peter, rising up, said to them: Men, brethren, you know, that in former days God made choice among us, that by my mouth the Gentiles should hear the word of the gospel, and believe.
-- Acts 15:7
And of course it wouldn't matter anyways because there was a sizable Jewish population in Rome and Babylon, as others here have noted, was essentially abandoned. It is also well documented that Clement wrote in the last First Century of Peter's martyrdom in Rome as did Tertullian a century later.
The Scriptural source for the “not the Gentiles” canard is rather thin gruel.