Kudos for clobbering the cad. Nevertheless, your subjective assessment of what a man should and should not find attractive in the manner of dress is flawed in that by definition the behavior of others is not subjective.
But what was the old man doing with the frying pan, before he was clobbered?
(KIDDING, KIDDING. Not everything has to be taken at heart attack seriousness, even though it seems that way to us.)
I suppose an enlightened (as to context) response would be something like “unhand me, you are not my husband.”