Well, to be fair, in 1981 when travel from the USA to Pakistan was banned he did travel there under a foreign passport.
And then there are those pesky reports he went to Russia where he attended the KGB Charm School for sleeper agents and moles.
Oswald Scholarship?