I could say that a young woman with no kids telling us how to raise ours is like rich liberals who send their kids to private school telling us how to fix public schools...
Neither of which is in any way relevant..
Yes, you could say that all day, and it wouldn't make it true.
Here's another example.
I have never had an abortion. But I know what one entails, and I have researched its effects, such that I am comfortable saying abortion is bad, not just for me but for everyone.
By your "logic," the fact that I have not had an abortion disqualifies me from speaking about abortion in any way.
Is that the case? If not, why is my example inaccurate? After all, I'm prescribing a course of action - not having abortions - based on no experience of my own.