Absolutely correct. If you look at the report by the GAO, it hints at the costs outlayed for "criminal illegal aliens" (of course, being an illegal alien is, by definition, "criminal").
State Prisons
How many incarcerated:
• Fiscal year 2002—SCAAP reimbursed all 50 states for incarcerating about 77,000 criminal aliens.
• Fiscal year 2003—SCAAP reimbursed 47 states for incarcerating about 74,000 criminal aliens.
• 5 state prison systems incarcerated about 80 percent of these criminal aliens in fiscal year 2003—Arizona, California, Florida, New York, and Texas.
Country of citizenship:
• Data on citizenship of criminal aliens reimbursed through SCAAP [State Criminal Alien Assistance Program] not available. [This is because citizenship is NOT inquired into by the state/local jails....but that's a whole other story; see, for example: States/localities often do not know with certainty the citizenship and/or immigration status of inmates. On the other hand, States/localities are only required to verify foreign birth--not to query inmates about nationality or citizenship. Therefore, when in doubt, jurisdictions have an incentive to submit costs for inmates of questionable nationality...]
• In mid-2004, most of the foreign-born inmates for the 5 state prison systems with the most criminal aliens were born in Mexico (60 percent).
BTW, the President has cut from his 2007 budget, funding for the SCAAP.