Yes we did. There were a few fortifications that the British maintained control of for several years after the Revolutionary War. I suspect that this was largely due to the fledgling American nation recognizing the limits of their power - something that was tested again a few years later in 1812.
And again I ask you, under what pretext did Lincoln invade Mississippi?
I'm not sure where you're going with this...my recollection is that the first skirmish in Mississippi was when the rebs seized the fortifications at Ship Island, and the first engagement in Mississippi was when the union took it back.
Sorry, but the myth that Northerners LOVED the black man were just that, myths...