It would be great if you didn't need a permit to conceal a weapon. Then you would really have the choice...
Since Mississippi got rid of the snags that made its "Constitutional Open Carry" a farce for so long, I have carried openly on a number of occasions, mainly to test the waters of public reaction - so far no issues.
As far as the weapon going with a dress, see if you can't put together a getup like worn in one of the Annie Oakley flicks....
I’m a bit old for the Annie Oakley look—LOL!