Of course it is meant to be derogatory. Just like evangelical or Christian right, or fundamentalist. But remember, Christian was originally a derogatory term also.
I would rather be labeled a "natalist" than an abortionist.
Better "Natalist" than the homosexual derogatory term for hetrosexuals: "Breeder."