God made us physical with senses that are to be used. Our senses give Him glory when used according to His Will. Beautiful art lifts the spirit, just as ugly "art" can oppress it. Surely, whatever touches our senses and draws us closer to Him is good.
True. Only God can read hearts.
However, behaviors are a big clue.
If images of the saints are evil, then so are mirrors.
The "language" of reverence, including the extravagant words we use and the elaborate gestures, are confusing to some Protestants. Consider the Wren "Auditory" church, with Bible and pulpit given prominence and beauty confined to proportion and chaste decoration, with the clear artistic message that what MATTERS is the WORD.
Not that that's necessarily or always a bad thing ... and some of those buildings are lovely. St. Paul's Chapel in Manhattan is very fine.
So when they see us kneeling before a reliquary and kissing it, they think it must be idolatrous. Not being familiar with this "language" and being strangers to our ecclesial culture they still think they can use our stylized gestures as a window into our hearts and judge us guilty of idolatry.
It's a kind of cultural provincialism. It's similar to those cultures where it's considered rude NOT to be talking while someone talks to you. Those of us not used to that will conclude these people are so self-absorbed that they never listen. While they, conversing with us, think our silence means we just don't give a hoot.
Our adversaries, who will stand up before their elders and will shake your hand, will scoff at any stylized gestures, not realizing how they also have a repertoire at hand. They disbelieve the notion of a courtesy foreign to their own, though they may know more about the bows of the Japanese than the genuflections of Catholics.
Further, a good maxim is "Lilies that fester smell far worse than weeds." We see, from time to time, over here a body-despising Jansenism, and over there a heretical cult of the Virgin. Our adversaries will jump on these and as much as say that because things that taste good might conceal a poison, we should eat only bland food for the rest of our lives, or that because some use the gift of sexual intercourse illicitly we should all be continent and that those who embrace their spouses must be libertines.
They are expert, in their minds, on our hearts. They will not hear us.