Yeah, you are right. Conquering the known world, regardless of cost, to have the Roman Empire rule all and they suddenly stop because, heck, it just ain’t worth it.
In fact the Romans campaigned well north of the Wall, sweeping all opposing forces from the field and withdrawing when their were no inhabited places left to take and burn and no remaining organized forces opposing them.
See Agricola and the battle of Mons Gropius.