That thought has crossed my mind many times.
Totally off -topic - but interesting.
Three recent Presidents who were abandoned by their fathers as infants/toddlers and raised by step-fathers.
Gerald Ford, Bill Clinton and B.Obama.
Newt would be the fourth.