Remember also that early on we were being told that Trayvon was a mere 140 lbs versus Zimmerman's 250 lbs. Then the forensic evidence shows the truth: 158 lbs for TM and 185-200 for GZ.
I think the reason why the height difference is important is that someone four inches shorter might have second thoughts starting a fight with someone 4 inches taller. And if he were to get into a fracas, the shorter person would be hard-pressed to land any punches. It's not just the height but the arms reach of a taller versus a shorter person.
That's why I think it's important those searching for the truth not play the same game and claim Martin was 6'3" and 7" taller than Zimmerman; it's the weight exaggeration all over again, only this time favoring Zimmerman.
However, I do agree with you that, in my opinion, Zimmerman would probably be less likely to start a fight with a younger man 4" taller than him. Martin supporters are going to scream "but Zimmerman had a gun!", as if there's evidence that Zimmerman had it drawn and was pointing it at Martin.