“It wasn’t KNOWN then, so how could it have mattered?”
It had to be known. If they moved close enough to Arthur’s birth for the birthplace to be in question, then they HAD to know Arthur’s father did not live in the US for many years prior to Arthur’s birth.
Chester's father was a Baptist minister. His mother was from the US. Chester was their fifth child. Their 2nd Child was born in 1824 in Vermont, while their first was born in 1822 in Canada. Chester was born in 1829. Thus his father had been in the US for at least 5 and possibly as much as 7 years when he was born. Naturalization took 5 years.
Plus Arthur claimed his birth year as 1830, giving his father yet another year to have become naturalized. There'd be no reason to think he had not. But he had not. Arthur also claimed his father immigrated when he was 18, and had lived in the US for several years before being married, but he actually did not immigrate to Canada until he was 22 or 23, and was at least 26 before he came to the US. So, at the time Arthur was runnign for VP, it was believed his father had been in the country much longer than he actually had, but even that much shorter time was still enough to have been naturalized before Chester was born.
The supposed foreign birth was a red herring, and distracted the public, the reporters and even the opposition, from examining his father's citizenship status.
And of course his father was know to be a citzen at some point, which he was.