...”He’s from Kentucky, that’s the way people talk.”...
That’s the way I read it. Further, I would be willing to bet that the “context” - which is what Obama constantly talks about as important - was that of a roomful of southerners, who use that term in a non-racist way to describe males in general.
I can’t stop giggling over the Foghorn Leghorn voice that I imagine him using to describe Obama. “Boy, I say, Boy”