Would you seek marital or sexual guidance/advice from someone that is celibate?
Would you then refuse medical care and advice for a disease if your doctor has never personally suffered from it? Should cancer patients only accept help from doctors with cancer? Is that really a qualifier?
The relevant question is not this, but whether the person you are seeking advice from is knowledgeable about the subject and able to convey that knowledge to you. If a priest successfully lives a celibate life, or whatever level of chastity is relevant to their life situation, they sound like a good place to go for advice on also being chaste. These are people who, theoretically at least, sacrifice worldly pleasure for Christ, and you wonder why a Christian would ask them for guidance in matters of morality?
You realise those aren’t really very good comparisons, don’t you?
A doctor has experience/knowledge in treating diseases. What kind of experience/knowledge has a celibate had with marriage or sexual difficulties?