This is probably apocryphal, but I remember it from my school daze: Einstein was once asked by a student how many feet were in a mile and he responded, “Never memorize something that you can look up.”
I remember that... physicists think that way. Richard Feynman said the same.
However you are talking about geniuses. There is a lot they don’t have to look up.
Memorizing timetables, quotes, Bible verses, subway stops, a musical instrument and its scales. all grow and work the brain cells. Establish more connectivity. So does printing then handwriting.
One of the reasons the educated Asians have a higher average IQ than the rest of the world is their character based language. Thousands of characters to be memorized to be literate. It grows the brain.
Einstein was never in jail with no phone numbers memorized... Bet he would have thought twice before saying that... lol