I have heard the same story and I don't doubt it. There is a to this day a large Irish subculture in New Orleans that dates back to that time. I have talked to some and they sound like they are from Brooklyn.
I'm not sure that it was the 'death' issue however that lead immigrants to those jobs. Draining those swamps were public work projects and the government would either have to hire slaves from their owners or hire "freemen". Since sugar and rice crops were so profitable at the time it's doubtful that many plantation owners would forgo a crop to rent their slaves out to the government, and if they had, they probably would have charged more (opportunity costs) than what the Irish were willing to work for.
It would be interesting to see what the real history of those stories are.