If Yebbie wasn't such a squish he would have said, "I'll tell you when I'm done, son."
Much as I don't like Jeb, I really don't like douchy little bureaucratic self-important napoleons either.
It sounded like they (Jebber’s staff) had let the Rotarian guy know, ahead of time, just how long Jebbie had to speak, per JEBBIE’s schedule.
So, likely that Rotary guy was doing Jebbie a favor.
If you watch til the end, you see Jebbie trying to eat his fruit cup and his son (?) comes over and tells him it’s time to go.