Yes, impotence is one cause for ‘annulment’ because it’s assumed that marriage ought to be open for children.
“I’ve never seen anything in the Bible that convinces me that those called to follow God should be celibate.”
You’ve not read Corinthians? Paul is very clear. “The unmarried man does not have his attentions divided and can devote himself entirely to God.”