Population is not a key factor in economic growth, productivity is. It isn't how many workers you have it's how much they produce. With the advent of automation, workers have become much more productive, so you don't need as many of them. At the same time a parasitic class has arisen that produces nothing, but waxes fat off the labor of those who do. It is the welfare recipients and the innumerable self appointed bureaucrats who cater to the wants of the indigent as well as spend all of their time devising more and more regulatory hoops for the rest of us to jump through that drag the economy down.
If you could magically ship all of the non- producers (including 95% of all government employees) to Quebec, you would find that the US population would drop severely, the USA's economy would increase, the economy of Quebec would be about the same, and the average IQ of both places would be greater.
For that matter if we could just give NY, NJ, MA, CT, RI, ME, and MD to Quebec the rest of us would be a lot better off.
I take it then, that you are in favor of free trade? Productivity is great, but ultimately you need consumers to buy what is produced. If you can send the non-producing consumers all to Quebec and still get them to buy your stuff, great.
You are throwing in additional factors, but you aren't debunking the population/economy correlation.