Very good point. Active Military take note.
I don't think it's just active military. If one has a CHL, then carrying a handgun for defense should be a "lawful purpose".
Unfortunately, I don't think the federal government views it that way. I've heard anecdotal claims that it did go to court, and a federal judge ruled against the CHL holder. But, I don't have an authoritative cite of the case to confirm it.