Practically speaking, since audio in that range is fairly directional, and you need a lot of power, the utility of such a jammer is questionable. It would of course probably drive away every rodent and dog in the neighborhood.
Some people suspect the sonic injuries that occurred in Cuba to our diplomats resulted from interaction between ultrasonic sources, so I would not encourage anyone to make such a jamming device at home, or expose anyone to such a strong ultrasonic audio field. It could be hazardous, so leave that kind of thing to research labs.
No, even if a microphone had response that extended to that frequncy range (very doubtful in a mic intended for voice surveillance), it will respond to all frequencies and a simple filter would remove your jamming signal leaving the voice intact. But no one interested in surveillance would consider using a mic with response higher than the voice range so it’s a moot point. In any case, it wouldn’t work.