So what’s the big deal with celibacy anyway?
Is it supposed to be more virtuous? Make someone more holy?
What’s the issue with sex that the Catholic church has that it exalts virginity and celibacy so much?
I would put it this way. Celibacy is a big deal because:
(1) It's following a way of life recommended, by Jesus and by Paul, to all who have this calling;
(2) It's giving good example to the younger generation: after all, all the singles are supposed to be celibate/abstinent until they marry, even if they don't marry until they're in their 20's, 30's, or 40's. Our society blares constantly that this is not possible. Nobody can live like that. We say "Yes, with the help of God's grace, and impelled by the love of Christ, we can."
(3) For that matter, it's a good example to those who experience same-sex attraction. Some of the public LGBT advocates are always saying it's so unfair for Christians to insist that gays embrace the "appalling" "burden" of long-term celibacy, and it's flat-out discriminatory! No, it's not: everyone is called to be celibate/abstinent unless and until they can get married to an eligible marriage partner: one who is of age, consenting, not already married to somebody else, not within prohibited degrees of kinship, and of the other sex. Same expectation for everybody. Historically, this might include scholars, soldiers, sailors, explorers, indentured servants and the like, and (in my grandmother's lifetime) even teachers.
(5) It was in this basis that Paul wrote he thought it was better not to marry, than to marry (although, better to marry than to "burn"!) If you will read the main section in 1 Corinthians 7, you will see he makes a very good case for dedicating oneself to the Lord and to the Mission of the Kingdom without competing priorities. It's a very practical and compelling argument, especially to whose who are called in this way.
(5) Jesus called it living like a eunuch "for the Kingdom of Heaven", and that suggests also the eschatological dimension. It is a powerful living witness that the time is short, this world is passing away, and bonds to this present life are tenuous.
Philippians 3:7-9
But whatever were gains to me before, I now consider loss for the sake of Christ. What is more, I consider everything a loss because of the surpassing worth of knowing Christ Jesus my Lord, for whose sake I have lost all things. I consider them garbage, that I may gain Christ and be found in him.
None of this denigrates Marriage, of course, which is a Sacrament and a living image of this Mysterium Tremendum, the union of Christ and the Church, one of the primordial blessings and commands for the species as a whole, and necessary for the continuation of the human race.
I myself have notice that social esteem for virginity and for marriage go up and down in tandem. A society which devalues virginity/celibacy, likewise devalues marriage. Modern libertines have the same, minimizing view of both consecrated celibacy and marital fidelity: namely, it's too constricting, meaningless, not to mention impossible, to live like that.
On the other hand, a society which greatly honors virginity/celibacy, also greatly honors marriage, and considers it high and holy.