There are a great deal of political cartoons from the US and GB in the 19th century that depict the Irish as part ape. And they were frequently referred to as a separate race.
I am not sure that whiteness counted for much back then
The Irish were apes until the north needed troops for their war on the south. The north, for their part, learned in the Mexican-American War that Irish troops should be led by their own officers; the embarrassment of having hundreds desert and fight for the other side (the San Patricios) wasn’t repeated 15 years later.