The Irish were apes until the north needed troops for their war on the south. The north, for their part, learned in the Mexican-American War that Irish troops should be led by their own officers; the embarrassment of having hundreds desert and fight for the other side (the San Patricios) wasn’t repeated 15 years later.
Don’t think the ape depictions ended with the Civil War.
And that is only in the US. The prejudice against the Irish continued much longer in GB.