I was really stunned by the number of well-known people Gardner has encountered in his life. Donald Knuth and John Conway as friends and houseguests? Whoa. And, you're right, those personal details are fascinating.
I believe John Horton Conway was introduced to a wide public through Gardner's Mathematical Games column in about 1970 or perhaps 1971, when Conway's Game of Life became a fad among the belt-mounted slide-rule-holster set. The Game of Life was a big, big influence on me, functioning to some extent to put me on the life path I've pursued to this day.
I vaguely remember a column or two about Knuth.
Three or four times a year Gardner would write devote a column to the adventures of Dr. Irving Joshua Matrix and his beautiful Amerasian daughter Iva. These he would write in the first person, mostly so he could personally enjoy the flirtations of Iva, who was always just out of reach.
(steely)