Referring to George’s father, not Mit.
George's father, Gaskell Romney, wasn't a polygamist either.
You have to go back to Mitt's great-grandfathers before you'll find polygamists. Using the Romney surname, that's Miles Park Romney, Gaskell's father and Mitt's great-grandfather. He had five wives (and thirty children).
But George Romney's father wasn't a polygamist.