"How does one misplace a castle for a few centuries?"
Well, we "lost" James Cittie (Jamestown) for a few centuries, and it's only 400 years old, so I guess it's possible to lose a castle over a much longer period of time.
I lost my house, one night. Then I sobered up.