Of all the significant people in Japan at that time Yamamoto was probably the only one who knew that a protracted war with the USA would not end well for Japan. He lived in the USA for a while taking some of his advanced education. He recognized the economic and manufacturing potential that could be brought to bear in a crisis.
In general, the Japanese Navy was opposed to adventures that would bring war with the US. Many naval officers were well traveled and better educated like Yamamoto then their Imperial Army counterparts. They knew what the US could bring to the table of war if war was provoked.
There were actually quite a few high-ranking military personnel who were arguing that an attack on Pearl Harbor would be idiocy, although Yamamoto was certainly the most prominent figure.
Before the war, Yamamoto was prescient enough to foresee the US firebombing campaign against Japan, correctly predicting the firebombing of Tokyo itself.
Yamamoto went to Harvard.