I don't think WGA members can do so, without being expelled. That used to be the rule -- but it was changed. The occasionally funny actor Chris Elliott was a writer on Late Night With David Letterman back in the '80s. The WGA went on strike and Letterman stayed on the air. WGA member Elliott could write skits, but only if he also acted in them, hence Elliott's career in front of the camera.
Last time the WGA went on strike, or maybe the time before that, I checked, and that rule had been changed -- no writing at all, even if the writer is in the performance. Not sure if WGA members can just ad lib, without writing jokes/monologues.
I was going to ask if extemporaneous jokes and improv count.