Your point is good, we don’t want to worship rules, and this is also why it’s important not to be harshly judgmental but realize those doing abuses usually may simply not know better. But on the other hand, I think many Catholics are perturbed about these abuses not because they are being overly fastidious, but because the Mass is a prayer, a ritualized prayer where every action and symbol means something, and to disrupt that breaks the spirituality and distracts from the meaning.
For example, if you were saying the “Our Father” with your family, would you stop in the middle and say “Oh, by the way, what’s for dinner tonight?” Or even just from an aesthetic viewpoint, would you recite a beautiful poem by Shakespeare or Byron and interrupt halfway and rephrase it in your own words? Even though that may be authentic and heartfelt, it ruins the prayer or poem.
Darn it, my kids do! If we pause to take a breath ... in prayer, in the lectionary reading, in the Bible reading ... they erupt in random conversation! "Is it going to rain today? Can we go to the library? Bill took my underwear!"
Most abuses may be summed up in one word: irreverence. Ritualism, however, is an abuse of no less significance.