I challenge you to ask any parish priest if he feels alone. We don't call him 'Father' for nothing. My pastor's great grandfather was a married priest (Maronite Catholic Church) but the great-grandson chose celibacy. In Lebanon, the seat of the Maronite Catholic Church, half of the priests are married but, according to the Patriarch (Cardinal Sfeir), married priests solve one problem while creating a new one. It is a challenge to match a married priest to a parish, especially in contemporary society, when the priest depends on the parish for his total support. That includes lodging, education, medical & dental insurance for him, the wife and children. It is not uncommon in these parishes with married priests for the parishioners to scrutinize the lifestyle of the priest and his family - knowing full well that they are footing the bill. If the priest's wife shows up at church with a fine coat while the parishioner's wives are wearing something more tattered, the contributions take a big dip.
For this reason, only the celibate Maronite priests are sent overseas to minister to small parishes with big demands. Our pastor is much loved and on the receiving end of many invites to dinners and social events. But first and foremost for him is his ministry to us as pastor and guide. The most frustrating aspect of parish life for him is when his parishioners shun the Sacrament of Reconciliation or fail to show up for Mass on Sundays.
He is perfectly content with his chosen life which is obviously not the one you were intended to have.
And there are thousands of married Protestant ministers who function very well with wife and kids along. If a single Pastor comes along, ALL of the busybodies (you know the type :>) will attempt to marry him off to one of the single ladies.