By your argument I shouldn’t take fertility medicine to try to conceive, either. Yet I have a feeling that a lot of people who have your point of view would look at me now, with only one child in 5 years of marriage, and assume that I’m contracepting.
There's an exciting new approach in fertility health called Napro Technology (Link) which aims for a much more profound understanding of optimal male/female health, in order to rebuild or strengthen the natural procreative function. I hope you might find this interesting,and helpful.