The truth is probably something else — the Saxons were probably already in Britain, and he joined forces with them in one of the internecine wars the led to the steady downward spiral of Celtic sovereignty on the island. One of the prospects for the historical King Arthur left his fortified capital with most of his army in order to fight and defeat one of the Germanic armies, and when he returned, a neighboring petty king had swung in and burned the city to the ground. IOW, I wouldn’t have blamed Vortigern one bit. :’)