I don't think Quebec had its own provincial assembly at the time of the American Revolution, either. French Canadians had been defeated and humiliated and were grateful to be allowed to keep their property, their legal system, and their religion. They weren't going to risk all that and be defeated all over again. Nor were French Catholics and New England Calvinists willing to throw in with each other.
Nova Scotians wished to remain neutral. There's plenty of speculation about why they didn't join in America's revolution -- geographical isolation, more recent settlement, ethnic make-up, economic underdevelopment, fear of the French, fear of Britain, dislike of Yankees -- but after American privateer raids and British occupation, there wasn't much chance of its joining America's revolution.
Still boils down to the "Usurpations and Abuses" were tolerable from a Canadian perspective. BroJoeK is saying the founders had a right to leave because the abuses they suffered were intolerable. The fact that the Canadians remained with the Crown demonstrate that "intolerable" is a matter of opinion, and in the US Colonists eyes they were, but in the British and Canadian eyes, they weren't.
My point stands. What constitutes intolerable conditions is completely in the eye of the beholders. It is up to those who are suffering the abuse to decide if it is tolerable or not.