No, I, too, believe it was the failed attacks of the torpedo planes that shook up the Japanese. They, as well as Hitler, in Europe did not believe that the weak, spoiled AMERICANS had the courage and guts to fight with the tenacity and courage shown by the torpedo bomber pilots of Torpedo 8.
Two entire squadrons were lost, with but a single surviror, who watched the battle from his life raft and was later rescued. At the time, American torpedoes were totally worthless. It was, though, the DIVE bombers that sunk the 4 Japanese carriers.
And it was also the torpedo bombers' attack that pulled down the high CAP, which helped clear the way for the dive bombers.