I've had the same thing happen to me. I lived in England for about a year, twenty odd years ago. On several occasions, complete strangers would say, "Oh, you're from California."
When I'd ask them how they guessed that, they'd just laugh and tell me that it was obvious by my accent. That flipped my noodle at first, because I'd never considered that we Californians had much of any kind of 'accent'.
Now that I'm in Texas, some people still correctly peg me as being from Cali.
I don’t get it. The midwest has a detectable accent, and of course the south, but California is especially noted for not having an accent
He didn’t correctly guess where I grew up. His guess was a location 3,0000 miles from where I grew up and happened to be where my parents grew up. My parents’ dialect came through to the next generation. And it wasn’t a strong accent, like a Texas or New York or New England accent.