My surname line comes from the Pictish area of Scotland. We held manors and some other buildings in areas that trace back to Pictish known history. Of course long after the Romans the Vikings did a number on the Pictish Kings, they certainly did not kill all the Picts. Mostly the Picts in the far north were wiped out.
The Romans didn’t keep getting their butts handed to them, there simply wasn’t anything worth the cash and effort n of Hadrian’s Wall. In a way that may have been a consequence of Trajan’s conquest of Dacia, which was the largest single payoff in Roman history (the gold mines). His successor was Hadrian, who wanted to bang little boys in the butt undisturbed, and had to be talked out of retreating from Dacia, but who did retreat from the Persian Gulf, built the Wall. And once the wall was up, the frontier was quiet, other than the considerable economic activity. After the Romans left, part of the wall was used as a quarry for building stone. Most of the rest of the wall had been made of turf, as the Antonine Wall had been some miles to the north.