And I believe that an accurate interpretation of the statement is that he is possibly a suspect. After all, the cops went to find him as the owner of a car involved in a hit-and-run accident and they exchanged words and got into a scuffle. If he were the victim of the accident, I find it very unlikely that that would have happened.
Let me posit a scenario.
Let's assume, just for the sake of this scenario, that his car WAS the car that got hit in the hit-and-run.
Ivory, being a police officer himself, knows how investigations on this subject are to be run.
When the other officers arrive Ivory is already agitated, seeing as how his car has been damaged. He doesn't think the officers are doing their job correctly and starts telling them about it.
The officers tell him to sit down and be quiet, he doesn't know what he's talking about, thinking him but another civilian who DOESN'T know what he's talking about and is just being a blowhard.
Heated words are exchanged and they take him down thinking he may be a hothead that's going to blow.
He also could have been a suspect if his car had committed the hit-and-run and he may not have known anything about it. Someone else may have been driving.
The police come saying you're the person who committed a hit-and-run, you're coming with us.
Ivory says, "Like heck I am. When was it committed. I wasn't even driving my car then."
The officers, instead of trying to ascertain the truth, which to be truthful isn't their job, take him down.
Either of these scenarios, IMO, is believable.