As for the sermon, I'd just as soon forget the whole thing. Even when they're good, they seem like an interruption.
Homilies are one of the important parts of the Liturgy of the Word.
Some people "tune out" the readings from the Bible. And then what they need is an interpretation of those readings and how it can apply to their daily life.
No, don't take away the homily.
Homilies are also one of the places where the priest can fulfill one of his vows, that of catechesis, teaching the world about Catholicism!!!!!!
(tongue in cheek) But can I send the 'bad' homilists to your church? LOL!