Yes you could say that or you could say it started when Zimmerman first say Martin or where Martin started to approach Zimmerman in his vehicle. All of the points could be considered contact. None of them are justification for a physical confrontation.
Once again “What information do you have that justifies one or the other person in the case to attack the other?”
The law does not require either to retreat. The law allows either one to use force to protect his or her own life. The law even allows lethal force if your life is actually in danger.
Martin would have been right to physically attack Zimmerman since it was Zimmerman admittedly following Martin and not the other way around.