Except that he won't be throwing away several thousands of pounds of aerospace hardware with every launch. Which do you think is more expensive, a pound of missile or a pound of fuel? By your logic it would be cheaper to by a new plane than load the fuel to taxi back to the gate.
If he can get the recycling down cheap enough to amortize the costs of the rocket — which will still wear out — over more than one launch, he’ll save some there. The cost of the payload per pound is what he’s trying to reduce, and he WILL have to use more fuel — and build a larger rocket — to get the same amount of payload into orbit as his competitors do. SpaceX has cut the cost of building the rockets and the price per pound to orbit, but has thrown away the rockets, just like everyone else.
The history of space flight has shown that there are no savings from recycling. The Shuttle in the middle of its run ate up over a half billion dollars to recover SRBs and rebuild/refuel them, service the liquid fueled main engines and refuel them, and the price was higher and higher as it reached the end of its years of service (and two of the orbiters were of course destroyed).
I’d prefer to have Musk running NASA, which he’d probably never agree to do. But it isn’t about who’s paying, it’s about chemistry and physics.