The Scots weren't in Caledonia until quite a while after the Romans were gone from Britain. They were in Hibernia / Ireland, just another tribe of yahoos, drinkin' and a-shaggin' and a-fightin'.
General and governor Agricola apparently planned to finish up the whole of the British Isles, which would have been the best move overall, as there was next to nothing beyond that could be used as a staging area. He began construction of a new capital (Chester, Cheshire, England). He was within a single campaign of finishing up in Caledonia when he was recalled by underachieving Emperor Domitian (last of the Flavians). Agricola may have built a (fortified) trade entrepot on the east coast of Ireland, and throughout that country, Roman stuff has turned up here and there in archaeological contexts and metal detectorist finds.
Emperor Hadrian was big on knockin' off young boys, and in all other ways living a life of leisure and privilege. After his death, Emperor Antoninus Pius expanded northward (he didn't go himself), having the Antonine Wall built (turf and timber), along with restoring some of Agricola's old camps, and building supporting roads. Control of trade and reducing the cost of border garrisons had been the point of Hadrian's Wall, and economically, the Antonine Wall (same purpose) reduced revenue and added costs, so it was abandoned less than ten years in.
Emperor Septimius Severus' was the final Roman bitch slap of the Picts, and he was an opportunistic and grandiose conqueror, proud of his (self-alleged) Carthaginian roots. His dynasty led to the Roman Empire's Crisis of the Third Century.
Ultimately, the Romans never conquered Scotland because there wasn't anything there of value. The Picts rarely caused any serious trouble and never mounted any great threat, which may be why even the Scots had little trouble with them.
And most of the time Britain, like most of the provinces of the Empire, didn't generate enough in taxes to support the troops and auxiliaries there. Most of the surplus to support the entire thing came from Egypt (and a good bit of that from trade, which was even taxed at a lower rate) and Anatolia.