Those that sided with the British during the American Revolution - sent packing. Those that remained neutral or sided with the Colonists - welcome.
I mean, there really is no evidence of just about any of them anywhere. I’ve been up and down the coast and nowhere, especially my own home, has much in the way of historic remains, much less PEOPLE.