Now there is another case of a band member leaving at the right time.
Peter Gabriel leaving Genesis, for example. No way he could ever top The Lamb Lies Down On Broadway with them. He went on and made his excellent first solo album, and Genesis went on to make their best album, A Trick of the Tail.
Make your mark and walk away.