Now that you brought that up... I have similar and faint subconscious hints that this may have happened to me when I was a young child. I have faint memories of dreams I had when I was five that now might actually fit that possibility. Dreams of being taken “somewhere else” in my sleep. Of course they could not be explained at the time, just strange dreams because I had no conscious understanding or knowledge of the UFO phenomenon.
But because of my memory curse these come back to me once in awhile but are fuzzy with detail. But when they do it is kind of like extrasensory perception and a “gut feeling” of possible truth that this may have been the case. It just has a ring of possibility concerning these dreams. But it could just be coincidence because we are always “taken somewhere else” in our dreams. But like you, I often wonder...
When you were young you had a dream. As an adult you associate the dream with physically 'taken somewhere else'.
As I said, pet theories constructed in your own mind.