Grant might fit that criterion although he did have an extensive military background. AFAIK, he never was elected to a government office. Maybe Eisenhower too. Both men, as military leaders had plenty of executive experience.
The military is part of the government. I didn’t say “legislative.”