He couldn't have been more wrong.
Lazenby was taking advice from the wrong people alright. I often wonder what Diamonds are Forever would have been like had he stuck around. His performance in OHMSS is under-rated by casual observers. I particularly like his athleticism and the scene in Teresa’s hotel suite. If Connery had been in OHMSS people would doubtless regard as a lock for best of the series. The better Bond films are generally the ones with strong plots and villains regardless of who was playing 007.
I agree... Lazenby had it going on, he needed to develop it.