“Would an apology remove my assumption?”
It would remove the assumption that you’re a rude jerk, at least for me.
Probably more than a few others as well.
Your assumption is correct, no removal required.