Well, for all intents and purposes, you are correct. NATO is essentially a security/military alliance. If you do not play a part in that side of things, you are effectively not a part of NATO. The French are damnable in the way they do things though. To them, it means something to only withdraw from one portion of the treaty, even if that is the most relevant portion.
It's kind of like one of the original states only ratifying one article of the Constitution but insisting upon being part of the US. That's the French.
Typical of the french, they don't want any responsibility, but want to poke their dirty fingers in NATO's business. Lovable qualities isn't it. Exactly like the demonrats modus operandi.