It is a good question though; why didn’t Cameron and his government cut back on Commonwealth immigration? Why doesn’t the current government do it?
I’m glad Brexit passed, and I’d like to see other nations leave the EU, but it seems mass immigration from Pakistan, Bangladesh, and Nigeria are much bigger long-term demographic threats than immigration from Poland and Eastern Europe.
Government took measures to curb the non-EU immigration, but they weren’t able to communicate it to the people.
Here is one article from a leftist Guardian describing these measures - https://www.theguardian.com/money/2016/mar/12/eu-workers-deported-earning-less-35000-employees-americans-australians
I believe a lot of people were fooled into thinking that brown skinned immigrants are there because of the EU, when in fact the EU immigrants were blond and blue eyed from Poland and letting brown people in was UK-s own choice.
Oh, and the lot of public sentiment had to do with the political correctness of the press and acceptable targets. When a Polish immigrant committed a crime it was okay to report the perpetrator as a Pole. When a Nigerian immigrant did that then it had to be reported as “a British man”. That created a situation where articles about negative effects about immigration could only mention Poles.