Actually, we have all the uranium we could ever possibly need (it's easily recyclable from existing stocks) and gold is an inert metal, useful as an electrical conductor, but not all that rare on Earth, at least for any useful purpose (I omit its role as a monetary standard, essentially an obsolete usage since early in the last century).
The fallacy in your argument is your assumption that for something to be valuable, we have to import it back to Earth. The key value of lunar resources is their location; they are already in Earth orbit -- on the Moon. As it costs thousands of dollars to launch materials into space from Earth's surface, something already in Earth orbit has intrinsic value merely by virtue of its already being in orbit.
Specifically, what's really valuable on the Moon is the recently discovered water ice deposits in the shadowed areas near the poles of the Moon. Water is much more valuable than its obvious role as something to drink; it is (literally) rocket propellant -- you break down the water molecule into its component hydrogen and oxygen through electrolysis, freeze both into their liquid form, and use it to fuel the rocket engines of an Earth-Moon space transportation infrastructure. If you can routinely operate between Earth and Moon, you can also access any orbit in between, including the economically important geosynchronous orbits and the strategically important high-apogee orbits used by our national security apparatus to electronically eavesdrop on our enemies and potential enemies. The ability to easily and routinely move throughout Earth-Moon space is a technical capability worth billions of dollars both to our economy and to our national security.
Right now, no one owns this lunar resource. The USA discovered it, but the first one to get there and start processing it, "owns" it. It ought to be us, not the Chinese.
Another under-acknowledged benefit of the moon is that a radiation safe environment can be created by covering a hole or digging a cave.
That's much cheaper than constructing an entire space habitat- which has to be abandoned during periods of high solar activity anyway.
Moon production of oxygen, simple aluminum products, glass, and food are early lo-tech possibilities.
Lack of hydrogen is the moon's killer drawback- it's value for fuel is so high it would be worth mining for, or even producing it by nuclear processes.
And if moon-supplied raw materials prices are lower by a factor of ten (a hundred even?) or more than Earth materials, space-based manufacturing will be no longer a fantastically expensive niche industry but would have mass-production possibilities.
Heck, the control of lunar-supplied materials would make a moon base so valuable, it's biggest expense could be militarily defending it against competitors!