First, I never said there weren't many factors affecting economic growth. Second, your Cuban example is flawed. Cuba collapsed when Castro took over, and since it isn't a free market situation it can only grow as much as Castro's boot heel allows.
Besides which, you can have an increasing population without much economic growth, but it is nearly impossible to do much growing when your population is in severe decline.
Population is not a key factor in economic growth, productivity is. It isn't how many workers you have it's how much they produce. With the advent of automation, workers have become much more productive, so you don't need as many of them. At the same time a parasitic class has arisen that produces nothing, but waxes fat off the labor of those who do. It is the welfare recipients and the innumerable self appointed bureaucrats who cater to the wants of the indigent as well as spend all of their time devising more and more regulatory hoops for the rest of us to jump through that drag the economy down.
If you could magically ship all of the non- producers (including 95% of all government employees) to Quebec, you would find that the US population would drop severely, the USA's economy would increase, the economy of Quebec would be about the same, and the average IQ of both places would be greater.
For that matter if we could just give NY, NJ, MA, CT, RI, ME, and MD to Quebec the rest of us would be a lot better off.
China has a one-child policy and it's economy is growing very fast.