Only around 7% of all Germans were Nazi Party members.
Being a Party member was by invitation only. Doesn’t mean that the others weren’t loyal National Socialists.