Me, too. It is one of those apocryphal references that keep getting cited on this topic.
That and this alleged travel ban to Pakistan. Try finding any evidence to support that one.
This whole thing is like one big game of telephone. Someone says, "Maybe Obama was born in Kenya" and the next person says "I'll bet Obama was born in Kenya" and the next person says, "Maybe his mom went there to visit and couldn't get back in time" and the next person says "I heard his mom went there and couldn't get back in time" and so on and so on.