If Martin was scared, he would have entered the town home (which he reached earlier, according to goiter-neck) instead of coming back to look for Zimmerman.
Given that the Skittles were on the doorstep...it appears he went back out...but the jury has to be told about those Skittles.