"They've done that twice, during the Revolutionary War and the War of 1812."
Actually three times, if you count the Confederate raid against St. Albans, Vermont which was launched from Canada.
It was alleged the Canadians were in cahoots (Great Britain being an unofficial ally and supporter of the Confederacy). Canada denied this, but refused to take any action against the raiders.
Actually 4. Your two, another in 1838 , American supporters of the rebellion and again in 1866 , the Fenians.