That turns out to not be true. First, if they are not guilty, they are not criminals, and second, if an arrest is made and then a person no-billed by a grand jury, the arrest record for that charge can be expunged.
I know someone that happened to. He was out more than $30K to defend himself and get his record cleaned up after a bad arrest during a nasty divorce.
/johnny
It depends ENTIRELY on the state. However for federal expungement, is a complicated and confusing process that falls into two categories - legal expungement and equitable expungement. Legal expungement is usually only available if the a court determines that your rights of due process were somehow violated in your arrest, United States v. McMains, 540 F.2d 387, 389 (8th Cir. 1976).
Equitable expungement is much more complicated, and as of now it's not entirely settled law. Some Circuits recognize it, others do not. Depending on where you were arrested, you may or may not be able to seek (or get more precisely) arrested records expunged just for not being convicted.