I agree, there was no harm done. It's just that if things are left open to abuse, someone will abuse it without question. They may feel justified in doing so in their opinion, but that doesn't make it right.
There will always be room for abuse. There will never be any "bright line" to protect us from bad people. Someone says "we're a nation of laws, not men." That's not totally true, for every democratic republic must rely to some extent on the good faith, acts, and intentions of its citizens and officials. I'd say that it's worked pretty well so far.