I actually find the claim questionable...given the tremendous political turmoil at many points along english history, it seems improbable, if not downright impossible, that it wouldn't have been removed from local control at some point or other, probably many times.
"it seems improbable, if not downright impossible, that it wouldn't have been removed from local control at some point or other, probably many times."
This would not prevent a determined family from regaining control of the ancestral seat. Some have put great store in such notions, in the past.
The remains found under the garden may well have been ancestors of the same family.