A guy named Elmer Ellsworth was the leader of such a group, in Chicago, and an acquaintance of the new president, Abe Lincoln. He brought his performing unit to DC, yo protect the President, when the states started seceding, and no one knew just who was going to do what. A hotel keeper, across the river in Virginia, flew a Confederate flag, and Ellsworth went over to tear it down. He was coming down the stairs with the flag when the hotel keeper shotgunned him. The hotel keeper was shot, in turn, and Ellis got a state funeral and became martyr to the Republic.
The flag itself is in the NYS Military Museum, and I expect BLM to burn it, momentarily.
I am very familiar with the Zouave companies. Pretty cool stuff. Those people were probably neighbors so to lose them all together is pretty shocking.