All Lincoln had to do was order them to leave. Simple.
But it belonged to the United States. Why allow the rebels to steal that property, too?