Indeed, Paul said that. But, you are taking it out of its context. Paul was expressing his desire that all Christians (saints) not have the burden of raising a family, during the stressful times in which he lived.
But, under qualifications for leadership, he made it clear that the test of a man was how well he managed his own household. No man was to be elevated to leadership, unless he had proven himself able to lead his own family.
The limit to one wife, was not necessarily about divorce, so much as the steadfastness required to maintain that familial relationship.
IN Christ's economy, we are all saints, and need no intercessory priesthood. We are priests, and heirs to the Kingdom. The local church ministry is necessary for the proper nurturing of the new converts, and for maintaining doctrinal purity, among the fellowship. Paul never spoke of a special priesthood (as in the man-made RC hierarchy), but he only that restated that all Christian believers are priests, able to approach the Throne of God, without any need for another intercessor. Jesus is our intercessor......
Paul said it, not me...
Titus 1:5-9
5The reason I left you in Crete was that you might straighten out what was left unfinished and appoint[a] elders in every town, as I directed you. 6An elder must be blameless, the husband of but one wife, a man whose children believe and are not open to the charge of being wild and disobedient. 7Since an overseer[b] is entrusted with God's work, he must be blamelessnot overbearing, not quick-tempered, not given to drunkenness, not violent, not pursuing dishonest gain. 8Rather he must be hospitable, one who loves what is good, who is self-controlled, upright, holy and disciplined. 9He must hold firmly to the trustworthy message as it has been taught, so that he can encourage others by sound doctrine and refute those who oppose it.