Maybe Trayvon attacked because he had seen something he wanted inside one of the townhouses and knew with the neighborhood watchman observing him, that he couldn't break in and take it.
Given that GZ didn't know Trayvon, the past conduct wouldn't give GZ any additional reason to be suspicious of him. But it would demonstrate Trayvon's predisposition to want to settle disagreements with his fists, and DD's testimony shows he wasn't happy about being watched. The lends credence to GZ's assertion that Trayvon started the fist fight.