The noise and the sheer insanity of the crowds.... I remember reading an account of a band meeting they had in 1966, they were genuinely frightened at the ferocity of the fan phenomenon, and George Harrison said “I think we must not do anything to encourage it” or words to that effect.
They became solely a studio band their last few years.
Of course by 1967, their fans were no longer the innocent screaming girls. They were all turning on and dropping out.
George Harrison was always a bad flyer. When they were flying to Portland, an engine on their Electra caught fire while it was on autopilot and the pilots were chatting with John and Paul.
He was the most reluctant tourer after that...