This time he is gone...he is bringing The X-Factor to America next year and will be on the panel for that show instead.
With enough money on the line something can always be worked out. Of course ratings have started to slip, so it might not be worth the money it takes to work things out.