I know a woman who received an annulment because her husband sterilized himself.
Yes, I think that voluntary and permanent lack of consent to children (on the day of your wedding or before) is grounds for annulment.
When I wrote about defective intent, that was not what I was thinking of.