Everyone else in that town from the doctor, the retailer, the policemen, to the swimming pool contractor are servicing that original wealth creation.
When the factory closes the entire town economics collapses. Because a service only economy is not self sustaining.
There are a lot of views on what true wealth is. Some say it's money, some say 'the sun in the mornin' & the moon at nite', others say the only true wealth is fearing God, and it goes on and on. Personally I've never understood the orthodoxy of the Church of the Holy Factory where the only true wealth is what comes out of a factory and nothing else matters.
Somehow it would make more sense to me that the Doctor's hours have more value than those of the factory janitor he heals and imho that's why the Doctor gets paid more. I'd have thought the HolyFactory zealots would at least honor the folks that built the factory in the first place but no, the orthodoxy is that those guys are evil.
Sounds goofy to me but hey, I can accept the fact that it's a way of life for millions.